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SUMMARY 


This is the first of four volumes which report the results of a 
simulator experiment in which crew performance in a traditional 
airline cockpit (McDonnell— Douglas DC— 9— 30) was compared to that 
in an MD-88, a highly automated ("glass cockpit”) derivative or 
the DC-9 family. In this experiment volunteer airline crews who 
were Qualified in one of the two aircraft . flew a predetermined 
scenario designed in the same manner as line— oriented flight 
training (LOFT) scenarios employed for training purposes by the 
airline. 

The purpose of this study was to examine jointly cockpit 
automation and social processes. Automation was varied by the 
choice of two radically different versions of the DC-9 series 
aircraft, the traditional DC-9-30, and the glass cockpit 
derivative, the MD-88. 

The goals of the study were achieved primarily by comparing 
various performance measures of the crews in dealing with the 
situations introduced by the LOFT, and particularly by examining 
the effect of the automation (two models of the same aircraft) on 
crew performance, coordination, and communication. Th® simulator 
runs were videotaped and examined in detail . In addition , two 
experts, the LOFT instructor, and a line qualified pilot-observer 
rated crew performance, and the crew rated their own performance 
and perception of workload. Errors in procedures were also 
recorded, analyzed, and classified as to severity. Attitude 
questionnaires were administered to all members of the volunteer 
group, those who flew the LOFT as well as those who did not. 

Results show that the performance differences between the crews 
of the two aircraft were generally small, but where there were 
differences, they favored the DC-9. There were no criteria on 
which the MD-88 crews performed better than the DC-9 crews. 
Furthermore, DC-9 crews rated their own workload as lower than 
did the MD-88 pilots. It should be noted that while both groups 
were about equal in total flying experience, the DC-9 pilots were 
more experienced in type, and it is difficult to separate out the 
effects of this factor. There were no significant differences 
between the two aircraft types with respect to the severity of 
errors committed during the LOFT flight. .The attitude 
questionnaires provided some interesting insights, but failed- -to 
distinguish between DC-9 and MD— 88 pilots. 

Further results from this experiment will be reported in 
subsequent volumes. A summary of the contents of Volumes II, III 
and IV can be found in Appendix 5. 
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I. 


INTRODUCTION 


A. BACKGROUND: AUTOMATION, CRM, LOFT, and AQP 

This airplane requires constant awareness as to who is doing 
what and this is easily broken down. The 757 requires as 
much, if not more, crew coordination than other aircraft. 


Human nature being what it is, it is very easy for a pilot 
(Capt. or F/O) to want to do all the tasks associated with 
flying the aircraft (programming the FMC, selecting 
autopilot functions, setting MCP altitude window) by 
himself, and not bother the other guy, or get it done faster 
than him, or do it just because "I'm the captain and I want 
to do it all." The pilot not flying (PNF) rapidly becomes a 
non— revenue passenger and either sits back and tunes out, or 
the situation becomes a little tense. It's imperative that 
this not happen, by stressing coordination during training. 


B — 757 pilot interviewees (from Wiener, 1989) 


Human participation in system operation has always been a mixed 
blessing: on the one hand, many systems would never have achieved 
operational status without the flexibility and resourcefulness of 
human operators (e.g. manned space flight) . On the other hand, 
errors and inappropriate actions on the part of human operators 
have caused untold numbers of incidents and many accidents. Some 
of these, such as the collision of two B— 747s at Tenerife; the 
capsize accident of the ferry Herald of Free Enterp rise at 
Zeebruge, Belgium; the nuclear accidents at Three Mile Island and 
Chernobyl ; and the chemical disaster at Bhopal , must be 
considered catastrophic. In each there were documented failures 
of information flow from machine to human, human to machine, and 
most important for this study, human to human. 

There are also several parallels between manned space flight and 
aviation, and one of them is the relative importance of 
technology and human factors. In the early days of both fields, 
technology was the primary limitation to mission success, and. 
human factors considerations were minimized because the crew 
members exhibited "the right stuff". The right stuff means that 
by superior skill, and perhaps daring, test pilots were able to 
overcome various adversities. Romantic as it may be, the right 
stuff is not an approach to air safety that we would recommend. 

The role of human factors was realized in World War II, starting 
with the recognition of the importance of the human's physical 
interaction with the system (e.g., viewing instruments, hearing 
alarms, etc.). However, technological and other changes in the 
late 1970s highlighted other aspects of human factors which were 
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not being addressed by the traditional approaches. Two of these, 
the impact of cockpit automation, and crew coordination and 
communication, are the subject of this study. 


From the days of World War II, the human factors considerations 
for such systems involved the human* s physical interaction with 
the system: viewing instruments, hearing alarms, reading 
instructions, reaching controls, shape coding to avoid mistaken 
actions, etc. Technological and other changes in the late 1970s 
highlighted other aspects of human factors which were not being 
addressed by the traditional approaches. Research directed toward 
airline operations in three important areas was initiated at the 
NASA Ames Aerospace Human Factors Research Division. The 
relationship between these areas is depicted in Figure I 1 below. 


1 . 


2 . 


3. 


Automation. Wiener and Curry (1980); Boehm-Davis , Curry , 
Wiener, and Harrison (1983); Curry (1985) ; Wiener C 1 ^ 85 ' a<s 
1988, 1989) and others brought awareness of the drawbacks as 
well as the benefits of flight-deck automation. 


Social Processes. Ruff ell Smith (1979); Cooper, White, and^ 
Lauftpy (1979) ; Foushee and Manos (1981) ; Foushee (1984) ; and 
Kanki and Foushee (1989) are only a few of the works 
describing the importance of social processes ( leadership , 
communication, crew interaction) in the aviation domain. 


Fatioue. Graeber*s research (1988) investigated the effect 
of fatigue, shift work, and related factors on subjective 
responses and human/ system performance. 


AUTOMATION CREW COORDINATION 



FATIGUE 


Figure 1-1. Graphic representation of three primary areas of 
research into air carrier human factors. The shaded area 
represents the study by Foushee, Lauber, Baetge, and Acomb 
(1986); the hash-marked area represents the present study. 
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For the last decade the research programs have concentrated on 
studying one of the main variables at a time. The early phases 
of these programs were characterized by problem definition, and 
all three programs found it difficult to define their problem 
even when considering only one factor at a time. By the second 
half of the decade of the 1980s the research had matured to the 
point where combinations of the main variables could be studied, 
and their interactions determined 

To date the work of Foushee, Lauber, Baetge, and Acomb (1986) has 
been the only investigation to examine two of the factors 
simultaneously: cockpit resource management (CRM) ana fatigue, 

depicted as the shaded area in Figure 1-1. The surprising 
results of this experiment emphasize the importance of crew 
coordination rather than fatigue. The present study explores the 
way intersection between automation and crew ^ coordination. 
There has not been, to our knowledge, any investigations of the 
intersection of fatigue and automation, although it is generally 
presumed that automation reduces fatigue. 

Automation 

In 1979, as a result of several committee hearings into air 
safety which identified automation as a salient, though little 
understood area, Congress mandated that NASA study the human 
factors issues of cockpit automation. Renwick Curry and Earl 
Wiener began their studies at Ames, first attempting to determine 
the nature of the problem, and later to produce guidelines for 
the safe design and operation of automatic equipment. An early 
publication (Wiener and Curry, 1980) provided an overview of the 
human factors problems, and some rudimentary guidelines. The 
following year they began a series of field studies, working with 
airlines and crews, examining actual line operations (Curry, 

1985? Wiener, 1985, 1989). 

These field studies have indicated that the basic assumptions 
behind automation are subject to question. Very briefly, it 
would appear that: 

1. Workload is changed , not reduced by the new 
equipment, and may simply be relocated in time, 
sometimes to the benefit of safety, sometimes not. 

2. Human errors are not eliminated, but their nature may 

be changed. In many cases, the errors may be more - 
critical: that is, automation may eliminate small 

errors and create the opportunity for large ones. 

3 . There are wide differences of opinion about the _ 
usefulness and benefits versus risks of automation in 
the minds of line pilots, and therefore wide 
differences in patterns of utilization. 

It also became clear in NASA's field studies that the type of 
equipment on board exerted an influence on crew coordination, 


4 



communication , and. resource management. However clear, this 
statement is based only upon observations by Curry and Wiener, 
not on any empirical investigation. The need to examine this 
observation empirically led to the present study. 

Cockpit Resource Management (CRMj 

In the 1970s, following a series of disastrous accidents in which 
the failure appeared to be not with individual performance of "the 
crew members, but with their performance as a team, the worldwide 
aviation community began to recognize that a crew must be viewed 
more than a group of highly skilled individuals, but as a close- 
knit two- or three-person team. 

A recent example can be seen in the report of a B— 737 which ran 
off the end of the runway at Charlotte (National Transportation 
Safety Board (NTSB) , 1987) . Such a phenomenon, the lack of crew 
coordination during an in-flight emergency, had previously been 
illustrated in the simulator experiments by Ruff ell Smith (1979). 
In this experiment, airline crews, faced with a simulated in- 
flight abnormal condition, often did not take advantage of the 
resources, human or inanimate, available to them, and as in the 
accidents mentioned, failed to work together as a team. Often 
the captains did not function in proper leadership roles, and the 
junior officers did not perform as effective subordinates. Over 
the last decade NTSB reports have frequently cited poor crew 
coordination and communication as contributing factors m 
aircraft accidents, and they have repeatedly urged the FAA to 
require CRM training for pilots (Kayten, in press) . 

Ruff ell Smith's findings led to a new thrust in the study of 
flight-deck human factors known as cockpit resource management 
(CRM) , pioneered by NASA scientists and university grantees 
(Foushee and Manos, 1981; Foushee, 1984; Helmreich, Foushee, 
Benson, and Russini, 1986). For an overview of CRM research, see 
Foushee and Helmreich (1988) . 

At the same time several airlines and consultants began 
developing CRM training programs for crew members, with the 
intention of schooling them in the art of effective management, 
communication, and crew coordination on the flight deck, and 
emphasizing team performance rather than individual prowess. 

Today CRM training is recognized and encouraged by the FAA, but 
is still not an FAR requirement. FAA Advisory Circular 121-51 
defines the need for CRM training and provides a theoretical 
framework for its development, but stops short of telling the 
airlines either that they must provide this training, or how to 
do it if they chose to. The circular wisely notes that the 
concept is still in the developmental stage, and more can be 
gained by allowing the carriers to develop new concepts , than by 
forcing them to follow a government-specified program. 

Most FAR Part 121 airlines in the U.S. now offer some form of CRM 
training, ranging from two or three day courses of intensive 
training to an hour or two during recurrent training. Part 135 
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airlines are also now beginning to develop programs. Numerous 
commercial vendors have entered the market with CRM curricula and 
materials, and several contract flight training centers now offer 
CRM programs to their customers. 

Advanced Qualification Program (AQP1 

While air carriers for the foreseeable future will not be 
required to offer CRM training, the emergence of the concept of 
Advanced Qualification Program (AQP) will provide a strong 
financial and perhaps quality incentive to do so. Under Special 
FAR (SFAR) 58, which governs AQP requirements, airlines may elect 
to present AQP proposals on a fleet-by-fleet basis to the FAA for 
approval , or to continue with traditional training as governed by 
FAR 121 Appendix E, F and H. The principal difference between 
traditional and AQP flight training lies in the s fact that under 
AQP, the carrier may train their pilots to a criterion of 
proficiency, rather than "filling in the squares” with fixed 
hours of practice or numbers of maneuvers. It is the obligation 
of the carrier, however, to demonstrate that the criteria have 
been met, and this includes CRM proficiency. 

Should an airline elect to train under AQP, a strong CRM training 
program and self-evaluation will be required. AQP offers the 
incentive of more flexible ground school and simulator training, 
and more effective utilization of training resources, hopefully 
resulting in a higher quality product at lower cost. This alone 
will probably accelerate the pace of development of CRM training 
and evaluation in the U.S., and since much of the rest of the 
aviation world usually follows the lead of U.S. m the training 
technology, other nations as well. 

Line Oriented Flight Training (LOFT) 

Concurrent with the development of CRM was the emergence of a 
simulator training concept known as Line Oriented Flight Training 
(LOFT) . (The more general term, Line Operational Simulation 
[LOS], is now being used by the FAA. LOFT is considered a sub- 
category of LOS). The FAA's view of LOFT is covered in Advisory 
Circular 120-35B. A review of LOFT can be found in Orlady and 
Foushee, 1987. Wiener (in press) discusses the design of LOFTs 
for high technology aircraft. 

In traditional flight training conducted in simulators, emphasis 
was placed on individual skilled performance, primarily in 
handling abnormal situations: e.g. the ability to perform 

difficult maneuvers and to cope with emergencies. While handling 
the emergencies clearly required team effort of the crew members, 
little emphasis was placed on team functioning, leadership, or 
role definition: they were viewed merely as necessary elements 

in successfully completing required procedures. 

The CRM movement was largely responsible for the development of 
the LOFT concept. Training specialists came to recognize the 
need to emphasize teamwork and leadership concepts by giving the 
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crews the opportunity to fly an entire leg which would simulate 
as closely as possible the typical airline mission. Crews would 
be evaluated and debriefed not only on maneuver completion, but 
also on their ability to work together as a team. Debriefing 
with the aid of video tapes recorded during the flight were most 
revealing to the crews. Thus CRM and LOFT were complementary 
movements in air safety, CRM emphasizing the team performance 
concept, LOFT offering the opportunity to practice it. 


B. THE RATIONALE FOR THIS STUDY 

What the authors asserted in planning this research is that the 
modem, high technology ("glass") cockpits may require some 
special considerations . The assertion is that the equipment may 
drive a social atmosphere that is distinct from the traditional 
cockpits, and hence the interaction of automation and CRM must be 
considered. If this could be demonstrated to be true, at least a 
portion of a CRM training program might be tailored to a generic 
advanced technology cockpit. It should be noted that already 
several airlines have added, or are in the process of preparing, 
CRM instruction with respect to the high technology aircraft , 
either in formal CRM training or in recurrent training. 


There is now good reason to believe that automation and social 
processes in the cockpit influence each other, in both airline 
and space flight operations. The field studies by Curry (1985) 
and Wiener (1985, 1989) on the impact of automation have revealed 
the importance of crew coordination in avoiding many of the 
"traps" posed by the new equipment. We have noted as well that 
traditional role assignments, such as pilot-flying vs. pilot-not- 
f lying, even though spelled out by the airlines j operations 
manuals, often break down, particularly under high workload 
(Orlady, 1982; Wiener, 1989). 


Numerous B-757 pilots commented during interviews and in 
questionnaires m Wiener's 1989 study that there is a lack of 
obedience to the standardized procedures of "who does what , a 
problem usually not present in well— standardized traditional 
cockpits. This may not necessarily be bad; it could be an 
adaptation to the demands of the high technology cockpit, not 
anticipated by those who write regulations, procedures, and 
checklists, and provide flight training. But for the present, we 
must accept the doctrine that adherence to procedures, including 
a strict definition of "who does what" , is a basic foundation of 
cockpit safety (Degani and Wiener, 1990) . 

For example, the field study of B— 757 crews at two ma^ or airlines 
(Wiener, 1989) tentatively indicates that automation in these 
highly advanced aircraft may impact on crew coordination, 
supervision, and CRM in the following ways: 

1 . Supervision by the captain of the first officer may be more 
difficult: at the very least, it may be considerably 

different from that found in traditional two— pilot cockpits. 
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2 . 


3. 


For example, although standard operating procedure calls for 
the captain to review information loaded into the control- 
display unit (CDU) before it is actually executed, this step 
is often ignored in high workload situations. This can 
produce two undesired effects: a) insufficient redundant 

checking of information to eliminate errors? and b) 
insufficient situational awareness on the part of the 
captain, possibly resulting in the captain being surprised 
(a word often used pilot volunteers in this study) by 
maneuvers of the aircraft. 


There may be a de facto transfer of authority from the 
captain to the first officer due to the fact that many of 
the first officers are more proficient than the captains on 
the CDUs by which the pilots enter information into the 
flight management computers. As some interviewees have 
opined, "He who controls the CDU controls the airplane. 

This is not to imply an intentional usurpation of authority, 
but more likely an unintended by-product of the automated 
systems. This is also in part due to an equipment- induced 
impairment of the captain* s ability to supervise the flight. 

There is often a breakdown in the traditional '*who does 
what” (standard operating procedures) in the "glass cockpit 
aircraft, which is computer- induced. Pilots will often do 
tasks assigned to the other pilot, usually with his/her 
consent and awareness , for a variety of reasons . While ^ this 
may at times be effective cockpit resource management m the 
short run, in the long run it can be injurious to 
standardization, which is the foundation of safe piloting. 


C. OBJECTIVES OF THIS STUDY 

The purpose of this study was to examine jointly cockpit 
automation and social processes. Automation was varied by the 
choice of two radically different versions of the DC-9 series 
aircraft, the traditional DC-9-30, and the glass cockpit 
derivative, the MD-88. The research design was not intended to 
be a competition? it was not a question of which aircraft crews 
did "better”, but to enable an examination of the strengths and 
weakness of the two cockpit technologies in flying identical LOFT 
profiles. We felt that such a study is particularly important at 
this time for several reasons. 

1. This work may contribute to the technology base for not only 
transport aircraft, but also long duration space station 
operation. Many of those experienced in space flight feel 
that attention to automation and social processes are the 
two most critical requirements for a successful mission. 

2. This work may contribute to safer operation of present 
aircraft and to design of future aircraft where multi-member 
crews use advanced cockpit technology, and may provide data 
and examples upon which CRM programs can be tailored to 
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3 . 


modern aircraft. 

The advent of AQP, with its heavy emphasis on CRM training 
and evaluation , creates a need for a sound < foundation for 
CRM and LOFT program design. Since each aircraft type 
(fleet) must be presented with its own individual proposed 
training package, it would seem highly advisable to 
determine some of the differences between crew performance 
in traditional and high technology cockpits. 

Details of the research methodology can be found in the next 
chapter, and as well in subsequent volumes in this series (see 
Appendix 5) . 


D. ORGANIZATION OF THIS REPORT 

This report is organized in chapters designated by Roman 
numerals. Chapters I, II, and III deal with the background, 
purpose , and methodology of the study • The results are reported 
in Chapters IV and V. 

Figures and tables are designated by chapter number (Roman) and 
serial order in the chapter (Arabic) . Thus Figure III-2 would be 
the second figure in Chapter III. The 28 histograms reporting 
the data from the Likert attitude scale items carry their own 
serial number and thus do not have a figure designation as 
described above. 

References to other publications cited in this report are found 
in Chapter VII. We have made no attempt to offer a comprehensive 
review of the automation or CRM literature. A recent review of 
human factors in automation can be found in Wiener (1988) , and of 
CRM/Crew coordination can be found in Foushee and Helmreich 
(1988) . For a review of communication in aviation, the reader is 
directed to Kanki and Foushee (1989) , and Kanki, Greaud, and 
Irwin (1989). 

Further results of this experiment will be reported in subsequent 
volumes, listed below. A brief summary of the contents of 
Volumes II, III, and IV can be found in Appendix 5. 



EXPECTED 


VOLUME 

PUB. DATE 

TOPICS 

II 

Early 1992 

Further analysis of errors; 
analysis of procedures 

III 

Early 1992 

Analysis of communications 

IV 

Mid 1992 

Summary of first three volumes 
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E. SUMMARY 

This chapter outlines the need for a study of the influence of 
cockpit automation on crew communication and coordination, and 
the goals of this study. The motivation for the study was that 
previous field studies of automation had observed, but had not 
experimentally validated, the fact that communication patterns in 
the modern, high-technology aircraft appeared to be different 
than in the traditional cockpits. 

Another factor motivating this study was the fact that research 
on the individual topics of automation and communication had 
matured to the point that they might be studied in combination. 
Rapid changes are taking place in training technologies and 
regulations that might make it advisable to recognize . the . need 
for certain differences in training pilots for communication and 
coordination in the modern versus traditional cockpits. 
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II. 


CHARACTERISTICS OF THE VOLUNTEER PILOT GROUP 


A. OVERVIEW 

This study was based on data received from pilot volunteers at a 

cooperating U.S. airline. The three sources of information were: 

1. Biographical and flying experience data from the one-page 
sign— up sheet by which the pilots volunteered. A total of 
84 pilots was recruited by this method. A copy of this form 
can be found in Appendix 3. Two joined the study later. 

2. Questionnaires were mailed to 86 volunteers in January 1990. 
73 questionnaires were returned. The questionnaire 
contained both biographical and flying experience 
information, a 28-item attitude scale, and other questions 
pertinent to the study. Eight open-ended questions were 
also included. A copy of the questionnaire can be found m 
Appendix 2. 

3 . A simulator experiment was conducted during the summer of 
1989 in which 24 crews (12 DC-9 and 12 MD-88) flew a 
specially designed LOFT mission. Two of the MD-88 runs were 
discarded from the data due to instructor error. The LOFT 
runs produced a variety of data, including flight evaluation 
forms filled out by a special observer, and by the simulator 
instructor, self-reports from the pilots themselves, and a 
variety of measures derived from video tapes taken during 
the flights and analyzed at NASA*s Ames Research Center. 
These data are discussed fully in Chapter III and IV. The 
rating scales can be found in Appendix 1. 


B. RECRUITMENT OF THE VOLUNTEERS 
Recruitment Procedures 

In preliminary meetings with airline management and the Safety 
Committee of the Air Line Pilots Association (ALPA) a recruitment 
procedure to obtain DC-9 and MD-88 volunteer pilots was 
determined. As a matter of convenience, only pilots from one 
base would be used, and the experiment would be open only to line 
pilots in those two aircraft. A brochure of the question and. 
answer format described the purpose and methods of the study to 
the pilots. The final page was the sign-up sheet mentioned 
previously (Appendix 3) . If a pilot wished to volunteer, he 
merely filled out the form, assigned himself an identification 
(ID) code, and mailed the form to the University of Miami. 

The list was provided to the airline* s simulator dispatch 
section, who scheduled the 48 pilots who actually flew the 
experimental simulator session. Of the 48 pilots, two were 
recruited by the scheduler on his own initiative from sources 
other than the original 84 volunteers. These two pilots were 


11 



later sent a copy of the sign-up package, and they returned the 
sign-up sheet and from that point on were indistinguishable from 
the other volunteers. A copy of the questionnaire (Appendix 2) 
was sent to each volunteer . 

Schedule compatibility of each pilot was the controlling 
in crew selection from the volunteer group, as all sessions were 
on the volunteer's own time. 

The figure below shows the distribution of the 73 pilots who 
returned the questionnaire, by aircraft type and seat. This 
distribution did not differ in any way from the distribution of 
volunteers. Several pilots sent back blank questionnaires with 
the explanation that they were no longer on the DC- 9 or MD-88, 
even though the cover letter that accompanied the questionnaire 
requested that all volunteers, regardless of their present 
aircraft seat, fill it out. 


Seat and aircraft type. 



MD-88 CAPT 


Figure II-l. Distribution by aircraft seat and type of 
the 73 volunteers who returned completed questionnaires. 


The questionnaires were sent out during January 1990, following 
the completion of the simulator experiment (August 1989) . 
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Confidential ity 


Volunteers were assured of confidentiality by a system of self- 
assigned ID codes. The original sign-up sheet contained the 
volunteer’s name, home address, and telephone number for 
administrative purposes, as well as his self -^assigned ID code. 

No other document contained any identifying information, only the 
ID code. The code was entered into a computer database along 
with the other information on the sign-up sheet , but has since 
been removed, so that no pilot can ever be identified from ^ 

code. The simulator flights were identified serially from 1 to 
24, and a separate computer file linked the run to the two 
pilots 1 IDs so that a match could be made between LOFT data and 
questionnaire data. This file has also been erased. 

The LOFT flights were videotaped by two cameras simultaneously. 
One camera, already installed in each simulator by the host 
airline for their LOFT debriefing purposes, recorded a panoramic 
view of the cockpit from aft of the pilots' seats. A second 
camera supplied by NASA recorded a portion of the instrument 
panel. Audio was recorded from lapel microphones worn by the two 
pilots, and a third microphone suspended from the ceiling of the 
simulator near the instructor's panel. 


Standard practice by the host airline was to use their tape for a 
debriefing session between the LOFT instructor and the crew. By 
agreement with ALPA, the tape is erased at the end of the 
session. ALPA granted the project team special permission to 
keep the recorded video tapes, based on the assurance that they 
would be promptly sent from the training center to Ames, and that 
no persons other than project staff would be allowed to see the 
tapes. Several pilots requested . copies of the tapes of their own 
flights; these requests were denied. 


Representativeness of the Sample 

Experimenters can never be sure of the degree to which a group of 
volunteers faithfully represents the population from which they 
were drawn. Are people who volunteer for a project, who are 
willing to give their time for no direct gain, attitudinally 
different from those who do not respond? We are not prepared to 
answer this question, for by its very nature, we have no 
information about the non-responding group. Nor do we have data 
about why the volunteers were willing to do so. 

It has been suggested that many of our MD-88 pilots may be from 
the low end of the seniority distribution within their group. 
During a brief period in 1989 new aircraft deliveries had slowed, 
and a n umb er of newly trained MD— 88 pilots were placed on 
reserve, and flew very little. Some persons in the training 
center suggested that a number of our volunteers came from that 
group, and were at least partially motivated by the desire to 
practice their skills by flying the "NASA LOFT . " 
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The same question of representativeness may be asked about those 
who returned their questionnaire versus those who did not. This 
is commonly known in sampling theory as "non- response bias. 
However, since our response rate was so high (73 out of 86) , it 
is unlikely that any such bias affects our results. 


In summary, we believe that it is safe to assume that the 
volunteer group was fairly representative of the DC-9 and MD- 8 8 
pilots existing in 1989, though it may well be true that like any 
volunteer group, they possessed some intrinsic motivation that we 
would not be able to define. 


C. FLYING EXPERIENCE OF THE CREWS 
Flying Time and Experience in Type 

Pilots were asked on their sign— up sheet to state the total 
number of flight hours they had accumulated at the host airline 
(including second officer time), and the number of months they 
had been in type, DC-9/MD-88. The results are displayed below in 
Figures II-2 and II-3 . The difference between the samples with 
respect to total flying hours was not statistically significant. 



Figure II-2. Total flight hours 
of volunteers at time of sign— up (n * 83) . 


From 1989 to the end of 1990, the MD-88 fleet^had nearly doubled, 
and increased by one— third in 1991. At the time of this writing 
(fall 1991) there are 85 MD-88s in the fleet. In contrast, the 
DC-9s are scheduled to be retired in 1993. Some have already 
been retired. 
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Figure II-3 . Number of months in type for 
DC-9 and MD-88 pilots at time of sign-up. 



Figure II-4. Number of hours in type for 
DC-9 and MD-88 pilots at time of sign-up. 
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These facts, coupled with the fact that for various reasons many 
pilots tended to stay on the DC-9 a long time, produced in 
volunteer group large differences in experience. As Table IX— 1 
shows, the MD-88 and DC-9 samples had essentially equal flying 
experience, but Figures II— 3 and II— 4 and Table II— 1 indicate 
that they had less experience in the aircraft. There is a simple 
explanation for the months— in— type data: it was almost impossible 
for MD-88 pilots to have much experience m type due to the 
newness of the plane. The greatest number of months reported by 
an MD-88 pilot was 24. For DC-9 pilots, it was 156 months. 


Some DC-9 and MD-88 first officers obtained those positions 
relatively rapidly, not serving the customary years as second 
officers in B-727 and heavier aircraft. At the time of the 
experiment the airline was enjoying an expanding demand for its 
services, and was receiving new aircraft (MD— 88, B— 757, and 767s) . 
The pilot work force was expanding rapidly. Also, for a variety 
of reasons, including seniority, bases, quality of trips and the 
like, many engineers were remaining in their seats, allowing 
rapid movement of new pilots into first officer seats. 


TABLE II-l. Statistics on total flying hours. 


MEAN FLYING EXPERIENCE 
OF VOLUNTEERS BY TYPE 



DC-9 

MD-88 

Signif. 

Level 

TOTAL TIME 
(HOURS) 

5569 

6482 

ns 

MONTHS 
IN TYPE 

30.4 

9.5 

.002 

HOURS 
IN TYPE 

1697 

437 

.0002 
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Migration Patterns 

In order to appreciate the complexities of human factors in 
airline operations, and to understand the differences between^ the 
two experimental groups, it is necessary to explore further the 
movement of pilots from seat to seat. On the questionnaire 
pilots were asked to place a check mark in a matrix of aircraft 
type-by-seat indicating each of the seats they had previously 
held at the host airline. Those data are displayed in Tables II- 
2 through II-5. Likewise they were asked to indicate the seat 
held immediately before the present seat. Those data are 
displayed in four tables, II-6 through II-9. 


The migration patterns displayed here are fairly typical, with a 
few noteworthy exceptions. Both the DC-9 and MD-88 captains 
showed the classic patterns, with most. haying served in the 
engineer* s seat of the three— pilot jets listed, and a variety of 
first officer's seats in 727 and larger aircraft, before making 
captain in a lower seniority plane. Of the 39 captains m the 
study, fifteen listed experience in the 757/767 aircraft prior to 
going to the DC-9 or MD-88; eight of the 24 DC-9 captains and 
seven of the MD-88 captains. 


Among the MD-88 captains, there were nine other captain 's seats 
listed, most notably five in the DC-9. Why would captains leave 
seats in 727s, 737s and DC-9s for an essentially equal seniority 
and salary aircraft? The answer probably is that they saw their 
future in glass cockpit aircraft, with the next stop being the 
757/767, and wished to start their exposure to the new technology 
as young captains, rather than waiting in an old technology 
cockpit for their chance to bid the 757/767. In contrast none of 
the DC-9 captains had previously held a captain's seat. 

The first officers likewise showed typical patterns, with most 
having served as engineers on 727s. _ Similar patterns can be seen 
in 757 pilots from two other companies (Wiener, 1989) . It is 
interesting that five of the 16 MD-88 first officers had 
previously held that seat in a DC-9. Their motivation to move to 
the same seat in the MD-88 was probably similar to that of the 
captains mentioned above. Transferring to the right seat of the 
MD-88 was essentially a lateral move, but it allowed early entry 
into glass cockpit technology. Their next step could be either 
the right seat of the 767, or to remain on the MD— 88, awaiting 
captaincy, and thereafter captaincy of the 767. Both of these 
would be easy transitions for a first officer well versed in MD- 
88 technology. 

The migration patterns seen in the DC— 9 and . MD-88 pilots at the 
host airline were generally typical of the industry for low 
seniority captains and first officers, with the added feature 
that the MD-88 was seen by many to be a "prep school” for a 
position on the 767. Likewise, the right seat of the 767 was 
viewed by many DC-9 and MD-88 first officers as a logical and 
advantageous position to be held prior to transition to captain 
on the MD-88. 
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TABLE II— 2 and II-3 . Seats previously held by DC-9 captains 
and first officers. 


SEATS PREVIOUSLY HELD: 
DC-9 CAPTAINS (n=24) 



Capt. 

F/Q 

s/o 

DC-9 

— 

21 

XXX 

MD-88 

0 

0 

XXX 

B-737 

0 

2 

XXX 

B-757/707 

flj 

8 

XXX 

B-727 

0 

17 

19 

DC-8 

0 

10 

6 

L-1011 

0 

15 

14 


SEATS PREVIOUSLY HELD: 
DC-9 FIRST OFFICERS (n=18) 



Capt. 

F/O 

S/O 

DC-9 

0 

— 

XXX 

MD-88 

0 

0 

XXX 

B-737 

0 

0 

XXX 

B-757/787 

0 

0 

XXX 

B-727 

0 

0 

16 

DC-8 

0 

1 

5 

L-1011 

0 

0 

1 
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TABLE II— 4 and II-5. Seats previously held by MD-88 captains 
and first officers. 


SEATS PREVIOUSLY HELD: 
MD-88 CAPTAINS (n=15) 



Capt. 

F/0 

S/0 

DC-9 

5 

" 12 

XXX 

MD-88 

— 

0 

XXX 

B-737 

2 

1 

XXX 

B-757/767 

0 

7 

XXX 

B-727 

2 

12 

10 

DC-8 

0 

8 

9 

L-1011 

0 

7 

8 


SEATS PREVIOUSLY HELD: 
MD-88 FIRST OFFICERS (n=16) 



Capl. 

F/O 

s/o 

DC-9 

0 

5 

XXX 

MD-88 

0 

— 

XXX 

B-737 

0 

1 

XXX 

B-757/767 

0 

1 

XXX 

B-727 

0 

0 

15 

DC-8 

0 

0 

1 

L-1011 

0 

0 

1 


L9 









































TABLE II— 8 and II-9 . 


Last seat held prior to present MD-88 seat. 


LAST SEAT PRIOR TO PRESENT: 
MD-88 CAPTAINS (n=15) 



Capt 

F/O 

S/0 


DC-9 

1 

1 

XXX 

MD-88 

— 

in 

XXX 

B-737 

1 

0 

XXX 

B-757/767 

0 

5 

XXX 

B-727 

3 

0 

0 

DC-8 

0 

0 

0 

L,— 1011 

0 

4 

0 




LAST SEAT PRIOR TO PRESENT: 
MD-88 FIRST OFFICERS (n=16) 



Capt 

F/O 

S/0 


DC-9 

0 

4 

XXX 

MD-88 

0 

— 

XXX 


B-737 

0 

1 

XXX 


B-757/767 

0 

0 

XXX 


B-727 

1 

0 

8 


DC-8 

0 

0 

1 


L— 1011 

0 

1 

0 
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Migration From the DC-9 and MD-88 


During the roughly 14 months between the sign-up of the volunteer 
pilots and the return of the questionnaire, a sizable number of 
the pilots left the DC-9 and MD-88 for other aircraft. Ten of 
the 73 pilots who returned their questionnaires indicated such, a 
move, with only one representing a promotion to captaincy. The 
old and new seats held by these 10 pilots at the time of the 
second questionnaire is summarized m Table 11-10. 


TABLE IX— 10 . New seats occupied by pilots who left the DC-9 
or MD-88 in the first 14 months of the study. 


NEW SEATS OF THOSE WHO 
HAVE LEFT THE DC-9/MD-88 


OLD SEAT 

NEW SEATS 

DC-9 Capt. 

727 Capt. 

DC-9 F/0 

727 F/0 
767 F/0 (2) 
MD-88 Capt. 

MD-88 Capt. 

727 Capt. 
DC-9 Capt. 
767 Capt. 
L-1Q11 F/0 

MD-88 F/0 

767 F/0 
L-1011 F/0 


There is no clearly discernible pattern in these movements. Five 
of the moves were to glass cockpit aircraft (four 767, one MD- 
88) • Furthermore it is difficult to understand some of the moves 
without knowing the motivation for the bid. In some cases the 
motivation may not have been an advance in equipment 
sophistication or salary, but bases or base security. Examples 
would be the DC-9 captain or first officer moving to the same 
seat on the 727. Although the 727 is a heavier aircraft, the pay 
differential is slight. The MD-88 captains who move to the 727 
and DC— 9 captain seats is somewhat in contradiction to what we 
have described as an upward progression through advanced 
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technology aircraft. In future questionnaires, we must ask the 
volunteers who have moved to state their reasons for the new bia. 


D. SUPPLEMENTAL INFORMATION 

Volunteers were asked for information regarding advanced 
aircraft, and their ownership of home computers. The exact 
wording of the questions can be found in Appendix 2. 

The table below speaks for itself. We were interested simply in 
knowing what each pilot thought was the most advanced aircraft he 
had flown. Note that the questionnaire made no effort to define 
"advanced" , leaving this to the pilot. 


TABLE II— 11. Responses to query about the most advanced aircraft 
(in pilot's opinion) he had flown. 


WHAT IS THE MOST ADVANCED 
AIRCRAFT THAT YOU HAVE FLOWN? 


AIRCRAFT 

MO-88 PILOTS 

DC-9 PILOTS 

DC-9 


2 

MD-08 

21 

1 

B-757/787 

9 

10 

B-727 


1 

DC-8 


1 

L— 101 1 

1 

12 

A-310 


1 

Mil. Transp. 


2 

Mil. Fighter 


12 


Pilots were also asked to name the aircraft in their company 1 s 
fleet that would be their first choice to fly if the quality of 
trips and salary were the same. Not surprisingly, the majority . 
of pilots from both fleets chose the 767. What is interesting is 
the number of DC-9 pilots (20 per cent) who chose the DC-9. 
Another 20 per cent elected the 727 and L— 1011. We cannot 
explain this result. It was clear throughout the study that the 
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DC— 9 pilots shared a great sense of loyalty to their aircraft. 

But it is somewhat surprising that about 40 per cent chose 
aircraft with outdated technology in response to this question. 

We can only speculate that among DC-9 pilots there is a group who 
are devoted to traditional technology cockpits, and wish to fly 
them as long as they can. Perhaps responding DC-9 or B-727 was a 
wav of exhibitina their disdain for the new technologies. 



Figure II-5. First choice of aircraft in company's fleet. 


Finally, the questionnaire asked the pilots about ownership of 
home computers. The data are displayed in Tables IX— 12 and Il- 
ls. These questions were designed simply to gather normative, 
data about computer ownership, as this is often stated as one of 
the reasons that first officers seem to have an easier time in 
glass cockpit ground school than the captains. Prior to this 
study we could find no data on computer ownership by pilots. The 
assumption has been that more first officers own home computers. 
This was not borne out in this survey. Half of the first 
officers, about three-quarters of the captains reported owning 
home computers. For this sample, the differences between 
captains and first officers were not statistically significant. 
There also was no significant difference in computer ownership 
between DC-9 and MD-88 crews. 
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TABLE 11—12. Number of captains and first officers reporting 
ownership of home computers. 


DO YOU OWN A 
HOME COMPUTER? 



YES 


TOTAL 

CAPTAINS 

28 


39 

F/Os 





45 

28 



TABLE 11-13. Type of computer owned by crew members. 


TYPE OF HOME COMPUTER 


TYPE 

CAPTAINS 

F/0# 

TOTAL 

None 

11 

16 

27 

IBM compat« 

11 

9 

20 

Macintosh 

4 

3 

7 

Apple 11 

9 

2 

11 

Other 

3 


3 

> 1 computer 

1 

2 

2 

Type missing 


2 

2 












What we failed to ask, however, may have been more important: 
who uses the home computer? Aside from the obvious economic 
advantage of the captains, it is conceivable that more captains 
own computers since DC-9/MD-88 captains are about the age to have 
children in junior high and high school, and we cannot presume 
that the pilot is the user of the home computer. Be that as it 
may, we have failed to show any true differences between captains 
and first officers, let alone link this to success m MD-88 
ground school. For now other explanations will have to be 
offered for the generally accepted fact that first officers tend 
to find ground schools in glass cockpit aircraft less forbidding 
than do captains. 
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III. 


STUDY METHODOLOGY 


A. OVERVIEW 

As explained in Chapter I, the fundamental purpose of this study 
was to determine the influence of cockpit automation on the 
performance of flight crews. The LOFT was designed to exercise 
the capabilities of the crews in a variety of areas, including 
fliqht guidance and navigation, systems management, monitoring, 
and crew coordination. The primary independent variable in the 
study was automation, varied by selecting two aircraft of the 
same family, the DC-9-30 representing a low level of automation, 
and its derivative, the MD- 88 , representing a high level of 
automation. The LOFT was written in such a W ^Y that it did not 
favor either aircraft, but should be equally difficult for both. 

The data reported here and in future volumes flowed from two 
sources: 1) the LOFT experiment? and 2) a questionnaire that 
volunteer pilots filled out regarding their attitude toward 
automation, and other aspects of their flying 30 b. The L OF T 
flights generated a large amount of data, only a small portion of 
which is reported in this volume. The remainder will be reported 
in Volumes II and III (see Appendix 5) . All of the questionnaire 
data are reported in Chapter V of this volume. 


B. EXPERIMENTAL DESIGN OF LOFT FLIGHTS 

In addition to the automation level , a second independent ^ 

variable which we had hoped to investigate was who was the pilot 
flying (PF) and pilot not flying (PNF) on each flight. The 
assignment as to which pilot would be the PF was determined 
before the crew arrived, and the experimenter asked the crew 
members in each case if they would agree to the assignment, which 
they always did. Thus the design consisted of two orthogonal 
independent variables, each at two levels, as depicted as a two- 
by-two matrix in Figure III-l. 

Aircraft Type 


DC-9 MD-88 


Captain 

Pilot 

Flying 

First Officer 



Figure III-l. Basic experimental design. 
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It was our intention to fly six crews in each cell for a total of 
24 crews, and this was done, but data from two HD— 88 crews had to 
be eliminated from the analysis due to departures from the script 
by LOFT instructors which invalidated the flight, yielding the 
unbalanced design depicted in Figure III-l. Though 24 crews 
completed their LOFT flights, two MD-88 crews (#16 and #20) were 
discarded from the analysis. In one case, the instructor 
inserted the wrong malfunction, then corrected it after the crew 
had detected the failure. In the other case, the instructor 
forced the crew to attempt to land at their original destination 
after the crew decided an approach would not be prudent. This 
leaves the design depicted in Figure III-l. In order to control 
for unequal sample sizes caused by deleting these two crews, the 
regression approach to calculating sums of squares was utilized 
in all analyses of variance. 


Thus all LOFT data reported here are baised on the 22 crews 
depicted in Figure III-l. Details of the LOFT scenario are given 
below and depicted in Figure III-2 . 

The choice of the PF/PNF did not turn out to be an effective 
variable. There was a point on the LOFT, as the crew began its 
descent to a Category II approach to Columbia, at which the 
captain had to take over as PF. For economic reasons, due to the 
high cost of maintaining training for Category II PF duties, 
first officers were not allowed to serve as PF on these 
approaches. Following the missed approach, some captains gave 
the PF assignment back to the first officer, others did not. 


C. LOFT SCENARIO 

The LOFT scenario design was a joint effort between the study 
team and the company *s DC— 9 and MD— 88 simulator instructors. It 
was designed to include periods of high workload, generated by 
system failures, deteriorating weather at the destination, and 
complex ATC clearances. The flight was based on a regularly 
scheduled flight for DC— 9s and MD— 88s from Atlanta (ATL) to 
Columbia, SC (CAE) . Due to low ceilings, a Category II approach 
was required at Columbia, and this was acceptable for both DC-9 
and MD-88 aircraft at this company. 

Figure III-2 displays the scenario for the study. The numbers in 
the figure correspond to the points that follow. (1) The scenario 
flown began at the gate in Atlanta. Weather was fogcyy at both 
Atlanta and Columbia. As the crew preflighted the aircraft, they 
had difficulty starting the auxiliary power unit (APU) which 
supplies electrical power on the ground or in the air if needed. 
The APU would start only after obtaining assistance from a 
mechanic. (2) Everything then went normally until the crew began 
their approach to Columbia. (3) At this point, one of the two 
constant speed drives (CSD) , which transfers power from an engine 
to an electrical generator, began to overheat, but the 
temperature leveled off in a cautionary (yellow) range, short of 
the emergency (red) range. This required that the crew monitor 
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Figure III-2. Simulation scenario 


AUTOMATION SIMULATION SCENARIO 



ATL Atlanta, GA 

A/C aircraft 

APU auxiliary power unit 
CAE Columbia, SC 

CLT Charlotte, NC 

CSO constant speed drive 

(generator) 


1 

Departure 

1 . Fog ahead of 
cold front 
causing 

low visibility in 
ATL (at alternate 
minimums) and 
CAE (CAT II). 

2. Dual alternates: 
ATL, CLT(VFR). 

3. At gate, A/C on 
ground electric 
power, APU, will 
not start until 
mechanic starts it. 


2 

Cruise to 
CAE 

1 . Normal, low 
workload 
period. 


3 

. Initial 
Approach 
to CAE 

1. Problem: left 
CSD temperature 
warning light 
illuminates. 

2. Checklist 
completion 
indicates 
approach and 
CSD operation 
can continue but 
CSD temp, must 
be monitored. 


4 

Approach/Miss 

Approach 

1 . Weather deteriorates 
requiring miss. 
Another A/C is in p 
published holding 
pattern so crew must 
enter non-published 
holding. 

2. CSD overheats and 
fails upon entering 
holding. APU will not 
restart. High workload 
begins. 


5 

Cruise to 
Alternate 

1. Crew must 
choose an 
alternate: ATL 
or CLT. CLT is 
best (ATL will 
deteriorate if 
the crew 
heads there). 

2. High workload 
period: 
completion of 
CSD shutdown, 
electrical off- 
loading, 
preparation for 
landing. 


6 

Landing 

1 . Relatively 
routine 
approach to 
Runway 5, if 
procedures 
have been 
completed 
effectively. 

2. Concern about 
possible failure 
of second 
generator. 
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Figure III-3. Approach plate for Category II approach 
to Columbia, SC. (Copyright Jeppesen Sanderson, Inc.) 
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the temperature after completing the associated checklist, rather 
than disconnect the CSD and its generator , and continue the 
approach to Columbia. (4) 


The weather was poor at the airport, and the crew ultimately had 
to abandon the approach at a low altitude. Because another no 
radio" aircraft had missed the approach immediately ahead of 
them, and could not be contacted by ATC, the crew had to fly an 
unpublished missed approach course to an unpublished holding 
pattern. During the missed approach, the CSD temperature began to 
climb to the red range, and if the crew did not detect this 
before reaching the holding pattern, the CSD failed, shutting 
down one of the generators. The missed approach threw the crew 
into a very heavy workload environment. They had to enter and 
maintain the holding pattern, secure the failed CSD and 
generator, talk to the company dispatcher and weather, and choose 
an alternate airport (5,6). Additionally, the APU, which would 
not start initially at the gate requiring a maintenance call, 
would not start at this point either, leaving them with only one 
operating generator. 


The crews eventually established themselves in the required 
holding pattern at CAE VOR, and contacted the company dispatcher. 
The dispatcher attempted to persuade the captain to return to 
Atlanta, a reasonable demand since Atlanta could provide adequate 
passenger convenience and aircraft maintenance for the airline. 
However due to the deteriorating weather at Atlanta, most crews 
elected to go to Charlotte (CLT) , which had clear weather (being 
north of the advancing cold front) , and was one of the alternates 
on the flight plan. 


This often was a period of intensive communication between the 
crew members, and with the Atlanta dispatcher. Various captains 
called dispatch and requested the weather at numerous nearby 
airports. Only one captain had briefed his first officer in 
advance of the approach at Columbia that if they had a missed 
approach they would file to Charlotte, thus making the decision 
in advance. Those captains who attempted to return to Atlanta 
were "forced out" by warnings from ATC of the deteriorating 
weather, and likewise for all other airports forward of the cold 
front. Although CLT was the obvious alternate, and was on the 
flight plan as such, it sometimes took some inveigling on the 
part of the LOFT instructor, acting as ATC or Meteorology, to 
head the crew there. 


D. DATA ACQUISITION 


Instrumentation 

All of the company's training simulators are equipped with a low 
light level video camera, as well as microphones that are worn by 
the crew during LOFT exercises. This camera is mounted from the 
ceiling of the simulator, behind the pilots . at about the position 
of the LOFT instructor. It shows a panoramic view of the cockpit 
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area, and the backs of the pilots' heads. The recorder is 
located outside of the simulator in an equipment bay. 

In addition, a NASA video camera was installed in the same 
location, recording the central instruments on the captain's 
panel. A microphone suspended from the ceiling near the 
instructor's station recorded his commentary as well. The LOFT 
instructor played the role of Maintenance, ATC, Dispatch, 
Meteorology, or anyone the pilots would contact by radio. He 
also played the role of the maintenance crewman who was sent to 
start the APU on the ground in Atlanta. All of this was recorded 
on a channel on the NASA video tape. The confidential handling 
of the tapes was discussed in the previous chapter . 

The tapes were viewed by qualified analysts at NASA Ames. They 
constructed a variety of logs of the actions of the crew , the 
errors made by the crew, and the verbal behavior. For purposes 
of this volume of the report, the creation of the error database 
from the tapes will be covered in the next chapter. More 
detailed information on the extraction of data from the tapes 
will be found in Volumes II and III. 

Observers 


Two observers in the simulator rated the performance of the crews 
following the flight. One was the company LOFT instructor, who 
filled out the usual CRM Evaluation Sheet (see Appendix 1) . The 
LOFT instructor also conducted a post-flight debriefing of the 
crew, according to the normal procedure for a company LOFT. The 
company's video tape was used, and self— critique by the crew, 
rather than that of the instructor, was encouraged. 

The other was the "NASA observer", a highly qualified company 
captain on medical leave, who was retained for this purpose. He 
had served as a line pilot and instructor on the DC-9, a line 
captain on the B-767, and at our request attended ground school 
on the MD-88. During the flight he took notes and filled out the 
Detailed Rating Form. Following the flight he filled out the 
Overall Rating Form, which can be seen in Appendix 1, and wrote a 
free text summary of the flight, detailing the strong and weak 
points of each crew's performance. These were used for the 
determination of the "most serious error" which will be discussed 
in the next chapter. 

At least one member of the NASA study team was present for each 
run. He greeted the pilots when they reported, attempted to set 
them at ease and emphasized that they should regard this as a 
normal company LOFT, and reiterated the confidentiality 
agreement. No volunteers expressed any concern about 
confidentiality. During each flight the study team member 
monitored the recorders, and observed the flight via the two 
video terminals outside the simulator cab. He also attended the 
post-flight debriefing, but did not provide any ratings or data. 
Following the debriefing, the experimenter again thanked the 
pilots and answered their questions, without revealing the full 
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purpose of the experiment, and asked them not to discuss the LOFT 
scenario with other pilots. We found no reason to believe that 
any crews arrived with prior information about the LOFT. 


Rating Forms 

Four rating forms were used in each flight. These can be found 
in Appendix 1. The data derived from these forms will be 
discussed in the next chapter • 


1. Each pilot independently filled out the Participant Survey, 
which was essentially a self-report on the workload and 
demands of the LOFT. This form is derived from the NASA 
Task Load Index (TLX) developed by Sandra Hart of NASA Ames 
(Hart and Staveland, 1988) . 

2. Overall Rating Form (one page) and a Detailed Rating Form 
(two pages) were filled out by the NASA observer. The 
Overall Rating form stressed crew communications, decision- 
making, management and coordination. These were essentially 
global scales of crew behavior, not related to particular 
duties or phases of flight. This form was derived from 
earlier work by Helmreich and Wilhelm (1987) . 

3. The Detailed Rating Form covered each step of the flight and 
each duty of the crew and each phase of flight, from 
prestart to approach and landing. This form was first used 
in a simulator study by Foushee, Lauber , Baetge, and Acomb 
(1986). 

4. Finally, the company LOFT instructor made his evaluation on 
a 17-item, five-point scale, ranging from "Poor to 
"Excellent", with "Normal" in the middle. This form was an 
adaptation by the company of the Line/LOFT form developed by 
Helmreich and Wilhelm at the University of Texas, and used 
by numerous airlines world— wide. 


Questionnaire 

An extensive questionnaire was mailed to the volunteers in 
January 1990, several months after the completion of the LOFT, 
experiment. The questionnaire, which is duplicated m Appendix 
2 , contained certain biographical questions on flying experience 
with respect to total hours and hours in type, types of 
approaches flown in their current type, a 28-item Likert attitude 
scale with probes regarding flying, safety, and automation, eight 
open-ended questions, and several miscellaneous questions. 

Results of the questionnaires are given in Chapter V, and 
elsewhere in this volume in graphic form. 
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IV. 


RESULTS OF SIMULATION EXPERIMENT 


A. OVERVIEW 

In this chapter we shall discuss the results of the simulator 
experiment, in which various measures of performance were 
recorded or derived. These included self— assessed measures of 
workload by the pilots? observer and instructor ratings of crew 
performance on the LOFT? and analyses of two independent sets of 
ratings by expert observers of the severity of crew errors during 
the LOFT. Additional analyses of crew errors will be found m 
Volume II. 


B. ASSESSMENT FORMS 

Measures of subjective workload and performance effectiveness 
were assessed on several forms that had been used before in NASA 
simulation experiments. Each of the forms used in this study is 
reproduced in Appendix 1. We will discuss them in the order that 
they appear in the appendix. 

Pilot Self-Assessment of Workload 

Each pilot assessed his own workload subjectively at the end of 
the flight using the "Participant Survey", an unweighted version 
of the NASA Task Load Index (Hart and Staveland, 1988). This 
rating form consists of six workload-related dimensions: mental 

demand, physical demand, temporal demand, performance, effort, 
and frustration. Pilots reported their responses along a seven- 
point Likert scale ("Low" to "High") with high numbers indicating 
greater amounts of the dimension in question, e.g. , a rating of 
"7" on, for example. Item No. 2, would represent a maximal self- 
rating of mental demand, etc. 

Hart and Staveland (1988) advocate the completion of a scale- 
weighting procedure wherein respondents evaluate the importance 
of each dimension to their perception of workload in the 
performance setting. However, in another full— mission simulation 
study, Chidester, Kanki, Foushee, Dickinson, and Bowles (1990) 
reported that a simple summed composite of the six items 
correlated .92 with the weighted scale scores. As a result, the 
procedure typically used to weight individual items of the TLX 
was eliminated and only the summed rating will be reported. Xn 
summary, each pilot circled a numeral from 1 to 7 representing 
his self -assessed value for each of the six dimensions (mental 
demand, etc.). These were summed to produce a mean for each 
dimension. 

NASA Observer 1 s Assessments of Performance 

The NASA observer was present in the simulator cab with every 
crew, and evaluated crew performance following completion of each 
flight segment using: 1. the "Overall Rating Form" , and 2) 

individual performance during specific phases of the segment. 
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using the "Detailed Rating Form." 


Overall rating form (crew performance) 

This form was adapted directly from Helmreich and Wilhelm ' s 
(1987) Line/LOFT Checklist, which was, in turn, derived from 
previous NASA simulation studies. The observer was instructed to 
complete the form evaluating the crew as a unit, not as two 
individual pilots. Ratings were made on five-point Likert scales 
and were intended to assess the expert observer * s overall 
impression of performance on each dimension. . Sixteen of the 21 
items on this form were analyzed. The selection of these items 
was consistent with the Chidester, et al. (1990) study, m which 
a factor analysis suggested these 16 items represented a common 
dimension. The remaining five items were excluded from the 
analysis • The numbers of the deleted items are circled m 
Appendix 1. 


These ratings were summed and averaged to form a crew— level 
composite for the flight. This means that for each crew a single 
number was derived that represented the NASA observer s opinion 
of the quality of the crew's performance. The new scale had a 
range from 1 to 5. 


The internal consistency of this scale as assessed by the 
coefficient alpha was .96. This indicated a high degree of 
internal consistency within the scale. Items composing this scale 
can be found in Table IV-1 and the form itself in Appendix 1. 

The scale will be referred to as "observer ratings of crew 
performance . " 


Detailed rating form ( individual pilot performance) 

This form was adapted from Foushee, Lauber, Baetge, and Acomb 
(1986) . Within each section of the form (pref light, taxi/take- 
off, climb, cruise, etc.) the observer rated the performance of 
both individual crew members. Ratings were on a 5— point Likert 
scale. 


LQFT Instructor 1 s Assessment of Performance 

At the end of the LOFT session, the company simulator instructor 
filled out the "CRH Evaluation Sheet." This was the only 
assessment made by the LOFT instructor. 

From the 15 items contained in this scale, 12 items were selected 
(see Appendix 1); they are displayed in Table IV-5. These 12 
items were selected from the 15 based upon a simple correlational 
analysis which revealed 12 of the items to be highly correlated. 
The remaining three items were dropped from analysis. Three 
measures were calculated based on the 12 items, and these will 
require some explanation. 
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Summary of Ratling Forms 

The fo ur rating forms that were employed during and after the 
LOFT flights are summarized below. Only the Detailed Rating Form 
was filled out during flight; the rest were completed during the 
flight. 


Form 

Participant Survey 
Overall Rating Form 
Detailed Rating Form 
CRM Evaluation Sheet 


Filled out by 
Each pilot 
NASA observer 
NASA observer 
LOFT instructor 


C. SELF-ASSESSMENT OF SUBJECTIVE WORKLOAD 


Method 

Following the flight, all crew members completed the Participant 
Survey (Appendix 1) derived from the NASA Task Load Index (TLX) 
(Hart and Staveland, 1988) . 

Self-assessment scores were analyzed by casting the data into a 
2 -by -2 (aircraft type-by-seat) analysis of variance (ANOVA) . 

This allowed statistical assessment of the influence of cockpit 
technology (DC-9 vs. MD-88) , duty assignment (captain vs. first 
officer) , and the interactions (non-additive combined effects) of 
these two variables. The role of PF versus PNF was not examined. 

Results 

The total composite score, a global measure of workload described 
in the previous section, the contrast between the two aircraft 
was significant at the .051 level (F (1,20) — 4.30). The other 
main effect (seat) and the two-way interaction were non- 
significant. To examine further the main effect of aircraft type 
(automation) , the results from the captains and the first 
officers were analyzed separately by t-tests for two independent 
groups (DC-9 vs. MD-88) . The results for the captains were non- 
significant, however for the first officers the difference 
between the mean composite workload ratings for the two aircraft 
were statistically significant, indicating that MD— 88 first 
officers attributed a higher workload to their cockpit duties 
than those assigned to the DC-9s. 

The mean workload score for MD-88 F/Os was 30.50, for DC-9 F/Os 
was 26.67. The mean composite scores for the four combinations 
of aircraft and duty position ("seat") are displayed below. 
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Figure IV-1. Mean composite self-assessed workload 
scores aircraft type and seat. 


Each of the six individual items that made up the composite score 
of the self-assessment form was also subjected to ANOVAs of the 
same design (type-by-seat). Of the six, two resulted in 
significant results in the contrast of aircraft type: Physical 

Demand (Item No. 3) and Frustration Level (Item No. 7) . In both 
cases, the means were higher for MD-88 crews than for DC-9 crews, 
meaning that MD-88 pilots perceived their jobs as being more 
physically demanding, and more frustrating. The means for the 
four groups on these sub— scales are displayed in Figure IV— 2 . 

In none of the six ANOVAs was the seat assignment, or the 
interaction of seat and aircraft type statistically significant. 
However, additional t-tests were run separately on the captains 
and first officers, testing MD-88 crews' rating against DC-9 
crews. These showed that in the two significant scales, for the 
first officers. Physical Demand was viewed as significantly 
higher for MD-88 crews compared to DC-9 crews, but for captains 
there was no significant difference. The Frustration Level 
ratings were significantly higher for both crew members in the 
MD— 88 compared to the DC-9 (see Figure IV-2) . The differences 
between the mean ratings were similar: t - 2.11 for captains, 

and t = 2.19 for F/Os. 
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Figure IV-2. Mean scores on the Physical Demands and 
Frustration sub-scales by aircraft type and seat. 


D. NASA OBSERVER'S RATINGS 



Method 

The NASA observer completed an overall rating form after the 
flight (see Appendix 1). This form was developed from the 
University of Texas Line/LOFT form in use at many airlines. 


Results 

The mean composite rating for the two types of aircraft was 
computed. For the 12 DC- 9 crews the mean was 3.43, and for the 
10 MD-88 crews it was 2.91 (higher ratings represented more 
satisfactory performance) . This difference was analyzed by the 
t-test for independent groups, and was found to be statistically 
significant (t = -2.40, p < .05). Thus in the eyes of the NASA 
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observer, 
performed 
88 crews. 


on a global measure of crew performance, DC-9 crews 
their duties on the LOFT more proficiently than the MD- 
though the absolute difference was small (17 per cent) . 


TABLE IV— 1* Sixteen items on Overall Rating Form administered 
by NASA observer. Note that these 16 items are taken from an 
original list of 21 items. 


CREW COMMUNICATIONS AND DECISION MAKING 

1. Communications were thorough, addressing coordination, 
planning, and problems anticipated. 

2. Open communications were established among crew members. 

3. Timing of communications was proper. 

5 . Active participation in decision making process was 
encouraged and practiced. 

6. Alternatives were weighed before decisions were make final. 

INTERPERSONAL (MANAGEMENT) STYLES AND ACTIONS 

8 • Crew members showed concern with accomplishment of all 
tasks at hand. 

9. Crew members showed concern for the quality of interpersonal 
relationships in the cockpit. 

WORKLOAD AND PLANNING 

11. Work overloads were reported and work prioritized or 
redistributed . 

12. Crew members planned ahead for high workload situations. 

13 . Appropriate resources were used in planning . 

CREW ATMOSPHERE AND COORDINATION 

14 . Overall vigilance 

15. Interpersonal climate 

16. Preparation and planning 

17. Distractions avoided or prioritized 

18 . Workload distributed and communicated 
21. Overall crew effectiveness 
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Detailed Rating Form ( Individual Pilot Performance) 


Method 

During the flight the NASA observer completed a second rating 
form that evaluated performance during each phase of flight (see 
Appendix 1) , the Detailed Rating Form. This form was adapted from 
Foushee , Lauber, Baetge, and Acomb (1986) • Within each section of 
the form (preflight, taxi/takeof f , climb, cruise, etc.), the 
observer rated the performance of each individual crew member. 
Ratings were completed using a five-point Likert scale and were 
summed and averaged to form a composite for each crew member 
during each phase of flight. Items forming this scale can be 
found on the Detailed Rating Form in Appendix 1, and will be 
referred to as "observer ratings of individual phase 
performance . " 

Results 

The two-page Detailed Rating Form can be found in Appendix 1. 
During the flight the NASA observer completed this form, 
evaluating performance during each phase of flight. These were 
s umm ed across phases, separately for two conditions. Normal and 
Abnormal. The first Normal phase began initiation of the LOFT on 
the ground at Atlanta and continued until the . generator CSD high 
temperature warning light illuminated on the initial approach to 
Columbia, marking the beginning of the Abnormal phase. The 
Abnormal phase ended when the crew began its approach to 
Charlotte, marking the beginning of a second Normal phase. 

The data were again aggregated across crew members, since we had 
abandoned the original plan of analyzing PF/PNF differences for 
the reasons stated in Chapter III . Also < we learned that . it was 
impractical to distinguish between captain and first officer 
proficiency, as their performance as a two-pilot crew was so 
highly interdependent. The ratings by the NASA observer are 
summarized in Table IV— 3 . The overall means, summed over the 
four phases of flight, are displayed in Table IV-2. 


TABLE IV-2. Mean proficiency ratings by NASA 
observer by phases of flight. 



Abnormal 

Normal 

N 

MD-88 

2.90 

2.96 

10 

DC-9 

3.23 

3.10 

12 
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The only statistically significant difference found in these 
analyses was in the overall performance measure. Here the 
performance of the DC-9 crews was again significantly better than 
the MD-88 crews (F(l,20) = 4.75, p < .05). However the 
differences were slight, the ratings of the DC— 9 crews being only 
about seven per cent higher than the MD-88 crews. These findings 
were consistent with the previous data showing small , though 
statistically significant, differences favoring DC-9 crews. 
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TABLE IV— 3 . 
of flight. 


Mean proficiency ratings by NASA observer by phase 
Larger n umb ers represent more proficient performance. 


Mean 


S.D. 


MD-88 



CC N 

3.02 

.15 

CC A 

3.07 

. 46 

SIT N 

2.97 

.16 

SIT A 

2.95 

. 47 

PRO N 

2.91 

.14 

PRO A 

2.75 

.33 

OVR N 

2.96 

.10 

OVR A 

2.90 

. 22 


O 

0 

1 

tO 




CC N 
CC A 


3.22 

3.36 

.59 

.63 

SIT N 
SIT A 


3.13 

3.19 

.16 

.60 

PRO N 
PRO A 


3.09 

3.04 

.24 

.48 

OVR N 
OVR A 


3.10 

3.24 

.20 

.47 


CODES 



N = Normal 

phases ; A =* 

Abnormal phase 



CC = Crew coordination/ communication 
SIT = Planning and situational awareness 
PRO = Procedures/ checklists/ cal louts 
OVR * Overall performance and execution 


Sample size is 10 for MD-88 crews, and 12 for DC-9 crews except 
in the CC-N where some data from one MD-88 crew was lost. 
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E. 


LOFT INSTRUCTORS 1 RATINGS 


Method 

One additional performance rating was completed by the simulator 
instructor. This utilized the LINE/LOFT checklist (referred to as 
a "CRM Evaluation Sheet" at the host airline? see Appendix 1) . 
This form is currently being employed through the airline s 
participation in the NASA/University of Texas CRM evaluation. 

There are some ambiguities in these data that can be attributed 
to the fact that we had a constantly changing instructor force, 
particularly in the MD-88. Ideally, for experimental purposes, 
we would have preferred to have had one instructor-evaluator for 
the entire experiment? the next best thing would be one DC-9 and 
one MD-88 instructor conduct all of the LOFTs. Neither of these 
was attainable for practical reasons. 


On the DC-9 side, two instructors ran all of the LOFTS? however 
on the MD-88 side, there were many instructors, and some observed 
only one or two crews. As a result, comparisons across observers 
were difficult. Each rating may contain uncontrolled random 
variation which cannot be expected to sum to zero. 


We made two attempts to "equilibrate" the instructors 
statistically. From the company's training records, we obtained 
each observer ' s average and standard deviation for crews observed 
in the CRM evaluation database. Based on those figures, we 
sought to standardize the present crew ratings. 

The following statistical adjustments could be made: 

1. We could analyze the uncorrected average of the 12 items. 

2. We could compute a "deviation score", the scale value minus 
the average rating each observer had given all company crews he 
has observed and rated prior to the experiment. This would m 
effect "calibrate" the ratings each instructor gave our crews by 
"unbiasing", adjusting for his tendency to give high or low 
ratings in the past. This would yield for each measure a simple 
score adjusted for the instructor's bias. 

3. We could create a "standard score" by taking the adjusted 
simple score (No. 2 above) and dividing by the standard deviation 
of all ratings the observer has given for company crews (a 
standardized score) . This would adjust not only for each 
instructor's bias, but for his variability as well. 

The simple score is just another overall rating. Deviation and 
standardized scores compare each observed crew to all others an 
instructor has observed and give us an idea whether he felt that 
each observed crew was better, the same, or poorer than the crews 
previously observed in LOFT. Thus the adjusted scores computed 
from the CRM Evaluation Sheets from our experiment would be 
directly comparable to regular company line crews who had been 
rated on the same form. 
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Results 


The three indices of instructor ratings were highly correlated: 
the correlation coefficients are displayed in Table IV-4. 

Also, the correlation between the instructor's rating (CRM 
Evaluation Sheet) and the expert observer's rating (Overall 
Rating Form) can be assessed. High correlations between the 
instructor and the observer would be the closest we can get to 
inter-rater reliability. Instructor simple scores correlated .80 
with the expert observer ' s ratings , while deviation scores 
correlated .43 and standardized .49. The simple score correlation 
is impressive, and suggests substantial agreement between the two 
evaluators of each crew. 

Each of the indices was submitted to analysis of variance. No 
contrasts were statistically significant. In brief, we could 
find no differences between the crews of DC-9s and MD-88s with 
respect to the LOFT instructors' ratings. The simple score came 
closest to discriminating the MD-88 and DC-9 (F (1,18) -2. 12, 
p=.167). The means are displayed in Table IV-6 below. None of 
the differences was significant. 


TABLE IV-4. Intercorrelations between simple scores 
and two adjusted scores on CRM form. 


Simple 


Deviation Standard 


Simple 


.65 


.71 


Deviation 


.95 


We should note two things concerning these data. First, simple 
scores approximate what was seen by the expert observer, but the 
effect (if any) was too small to be detected by our design. 
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TABLE IV-5. Twelve items (out of original 17) rated 

on LOFT form by the simulator instructor. 


Briefing thorough, established open communications, addresses 
coordination, planning, team creation, and anticipates problems 

Communications timely, relevant, complete, and verified 

Inquiry/questioning practiced 

Assertion/advocacy practiced 

Decisions communicated 

Crew self -critique of decisions and actions 
Concern for accomplishment of tasks at hand 
Interpersonal relationships/ group climate 
Preparation and planning for in-flight activities 
Distractions avoided or prioritized 
Workload distributed and communicated 
Overall crew effectiveness 


Table IV-6. Means for original and adjusted 
LOFT instructors* ratings. 



MD-88 

DC-9 

Simple 

3.163 

3.47 

Deviation 

-.08 

-.13 

Standard 

-.11 

-.42 
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Second , on the average, crews in this study were rated as 
performing less well than those crews that the instructor had 
seen in other company LOFT's. This second finding is not easy to 
interpret* It could mean simply that our scenario was more 
difficult than those typically used m the company s JLOFT 
sessions, which was probably the case. Comments made by both the 
instructors and the volunteer pilots indicated that this LOFT 
scenario was considered quite difficult compared to other LOFTs 
which the company employed. 

In summary, even with the statistical adjustments we made in 
order to attempt to compensate for the fact that we had a number 
of instructors rating the LOFT performance on the CRM form, we 
did not find differences in these ratings attributable to cockpit 
technology (DC-9 vs. MD-88) . More will be said of this in 
Volumes II and III of this report, which will analyze the 
communication between crew members in the two aircraft. 


F. ANALYSIS OF CREW ERRORS 

In this section we present data from two analyses of crew errors: 
1) ratings based on direct observation of the crews during the 
LOFTs by the NASA observer? and 2) analyses of the video tapes of 
each LOFT performed at NASA Ames by two qualified pilots who were 
trained to rate the error data. 

NASA Observer ' s Ratings — Most Serious Error of Each Flight 
Method 


During the actual LOFTs the NASA observer kept a log of errors 
that he had observed during the flight. He did not rate or 
classify these errors at the time. When all of the flights were 
completed, we asked the NASA observer to determine for each 
flight the most serious error committed. When he had done this 
for the 22 successful flights, we asked him to then rank order 
the 22 "most serious errors" from the least serious (rank 1) to 
the most (rank 22) . Thus he produced a rank-ordered list of the 
most serious errors from merged data of the two aircraft. We 
then analyzed these data using rank statistics for two 
independent groups (White rank sum test) . 


Results 


Our directional experimental hypothesis was that the MD-88 crews 
would commit errors which were more serious than those produced 
by the DC-9 crews. This was based on the observation by Wiener 
and Curry (1980) and Wiener (1988) that one of the effects of 
cockpit automation appeared to be that crews committed fewer but 
more serious errors. If this were the case, then the larger 
ranks would tend to be assigned to the 10 MD-88 flights and 
smaller ranks would go to the 12 DC-9 flights. The mean rank of 
the errors of the 12 DC-9 and 10 MD-88 "most serious errors per 
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flight” should be different, namely that the mean of MD-88 ranks 
should be larger, if the hypothesis were correct. 

In fact this was not the case. The ranks of the most serious 
errors were almost perfectly intermixed, yielding a Z of 0.20. 
Thus our hypothesis was not supported: the crews of the two 

aircraft types produced most serious errors of equal magnitude. 

Video Tape An alysis of Errors - Categories of Seriousness 
of the Error 

Method 

Videotapes recorded during the experimental LOFT sessions were 
sent to NASA-Ames Research Center for analysis. At Ames, error 
analyses were undertaken employing a complete review of the 
videotape records. Using these records, two highly qualified 
observers reviewed each flight for operational errors. Both 
observers were recently retired pilots . One retired from an 
airline with substantial experience line flying and instructing 
in the b- 767. The other retired from the military and had served 
as an observer in previous NASA investigations. Both observers 
studied the host airlines* s DC-9 and MD-88 training and flight 
operations manuals prior to initiating error identification. 

The observers worked together to review videotapes of each 
performance. When an error was recognized by one or both 
observers, the tape was stopped and the segment containing an 
alleged error reviewed. After this process, both observers had to 
agree that an error had occurred and had to agree on a 
description of the error or it was not counted in the analysis . 
This was a conservative error tabulation process that ensured 
that every error data point was reviewed at least twice. Since 
some performance errors were more operationally significant _ than 
others, errors were categorized according to level of severity. 
This process was accomplished by both of the observers involved 
in the videotape error analysis 

Error classifications 

A three-level classification, based on previous work by Foushee 
et al. (1986) was utilized. The three levels of error severity 
and their operational importance is listed in below. 

Type 1 — minor, with a low probability of serious flight 
safety consequences. 

Type 2 - moderately severe, with a stronger potential for 
flight safety consequences. 

Type 3 - major, operationally significant errors having a 
potentially negative impact upon flight safety. 
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Examples of Type 1 errors Included* unnecessary interruption . of a 
normal checklist, incorrect clearance readbacks that were quickly 
corrected, and missinq an item required during the takeoff 
briefing. 

Examples of Type 2 errors included: failure to complete a normal 
checklist not accompanied by significant missed items, and 
failure to check in with the tower in a timely manner as 
instructed by approach control. 

Examples of Type 3 errors included: troubleshooting a significant 
system failure (CSD) without using the written procedure, turning 
the wrong direction (reciprocal heading) on exiting holding, and 
either disconnecting the CSD before it was required or failing to 
monitor CSD temperature allowing overheating to shear its shaft. 

Results 


A total of 273 errors were identified by the video observers 
across the 22 crews. A consensus was reached by the two observers 
on the severity classification of each error. They replayed the 
tapes until they could agree on the severity. If they could not 
agree, the less severe rating was adopted. A tally of these 
errors by phase of flight (normal vs. abnormal) summed across the 
two aircraft types is displayed in Table IV-7 . The same data by 
phase and by aircraft type are shown graphically in Figure IV-3. 
The means, broken down by aircraft type, are displayed in Table 
IV-9. It is clear from the data displayed in Table IV-7 that the 
most severe errors (Type 3) prevail in the abnormal phases of the 
LOFT flight. 


TABLE IV-7. Error frequencies by normal and abnormal phases 
of flight, stimmed across two aircraft types. 



Normal 

Abnormal 

Total 

Type 1 

16 

13 

29 

Type 2 

51 

95 

146 

Type 3 

11 

87 

98 

Total 

78 

195 

273 


Error frequencies were correlated with evaluations of performance 
by the expert observer and by simulator instructors , as 
summarized in Table IV-8. Those which were significant were 
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negative, indicating that instructor ratings showed an inverse 
relationship to errors, as one would expect. An absolute value of 
r greater than .243 indicates significance at the .05 level (two- 
tailed) . 


TABLE XV— 8. Correlations between errors and observers* ratings. 


Error type 

Total 

1 

2 

3 

NASA observer 

-.45 

.18 

-.29 

-.62 

LOFT instructor 

-.46 

.04 

-.38 

-.46 


With the exception of minor (Type 1) errors, high error 
frequencies were associated with poor performance ratings. 

However, when submitted to analyses of variance, total error 
frequencies did not reveal significant effects for aircraft type. 

The data in the table reveal only minor differences between 
aircraft type in severity of errors. Based on an ANOVA, there 
were no differences between the DC— 9 and MD— 88 with respect to^ 
severity of errors in any category. This is consistent with the 
data produced from the examination of "most serious errors by 
the NASA observer. Though none of these comparisons was 
significant, they do tend to show MD-88 crews making more errors 
than DC— 9 crews overall • A case can be made that in the glass 
cockpit MD-88 there is simply more to do, and more errors can be 
made (e.g. attempting to reprogram the FMS while on the glide 
slope at low altitude) . Volume II will discuss the nature of the 
errors in the two cockpits. Our sample size is not sufficient to 
detect effects of this magnitude. We can only conclude that we 
have failed to produce evidence to support the hypothesis that 
more severe errors are made in highly automated cockpits. 

Errors by phases of flight 

As a follow-up to these overall analyses , error frequencies were 
broken down by phase of flight, to determine whether significant 
differences between aircraft might occur during some periods. 

Each tape was coded to identify eight phases: pre-takeoff, climb, 
cruise, descent, CSD overheat and procedure completion, approach 
to CAE (including missed approach and holding) , diversion, and 
approach to CLT. As might be expected, errors were very closely 
associated with the abnormal phases of flight. 

Of the 273 total errors identified, only 78 (29%) occurred during 
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normal flight (though these periods averaged approximately 50% of 
the total LOFT time) as previously defined as the period from 
initial cockpit activities in Atlanta up to theCSD failure light 
appearance at cruise between Atlanta and Columbia , and the period 
during the approach and landing at Charlotte. The means of the 
errors by error type (severity) , phase of flight, and aircraft 
type are shown in Table XV— 9, and Figure IV— 3. 


TABLE IV— 9 . Mean frequency of errors by error type (severity) , 

flight phase (normal vs. abnormal, and aircraft type. 

ERROR TYPE 


DC-9 

1 

2 

3 

Normal 

0.92 

1.83 

0.33 

Abnormal 

0.67 

4.50 

4.33 

MD-88 




Normal 

0.56 

3.22 

0.78 

Abnormal 

0.56 

4.56 

8.89 


ERRORS BY AIRCRAFT TYPE 
AND NORMAL/ABNORMAL PHASES 

LEGEND 
HI DC-9 N 
HI MD-38 N 

r/yi dc-9 a 

MD-88 A 



SEVERITY OF ERROR 


Figure IV-3. Operational errors by aircraft type, phase 

of flight. Normal (N) vs. Abnormal (A), and severity. 
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In order to ascertain what portion of the flights was spent in 
the normal and abnormal phases (as defined above) , the time was 
computed from the video tapes for each flight, and the means were 
determined. These means are shown in Table IV-10. An analysis 
of variance was performed on these data with aircraft type and 
phase as main effects. Neither was statistically significant. 

The mean time in the normal versus abnormal phases of flight were 
virtually identical. Although the mean time in both phases was 
slightly higher for MD-88 crews, this difference was not 
significant. The interaction term was also non-significant. 

Since the times spent in the normal versus abnormal phases were 
so nearly equal, the frequencies of errors can be viewed as error 
rates. One may convert these to mean errors per minute by 
dividing by the appropriate mean time (approximately 40 minutes) . 


TABLE IV-10. Mean time in normal versus abnormal phases 
of flight by aircraft type (in minutes) . 



PHASE 

OF FLIGHT 


Normal 

Abnormal 

DC-9 

39.8 

39.9 

MD— 88 

43.7 

42.0 

Overall 

41.5 

40.9 


Follow-up analyses were overall something of a disappointment. We 
had hoped that they would clarify the performance ratings by 
isolating where errors were more likely to occur in each air- 
craft. It appears either that these differences are very slight, 
or that the effects are so small that our design cannot detect 
them. In-depth analyses of the error data will be presented m 
Volume II of this report. 

The question remains whether the error data supports confidence 
in the observer ratings and self-reports of crew workload. Error 
frequencies reported here are highly correlated with independent 
ratings, which would tend to bolster observer validity, but they 
do not appear to provide independent evidence of operationally 
significant problems in the MD— 88. This will be discussed 
further in Chapter VI. 
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G. SUMMARY OF LOFT DATA 


The preponderance of data presented in this chapter points toward 
a consistent, but slight, superiority of the DC-9 performance 
over the MD-88. In the areas of self-report of workload, the MD- 
BS crews saw their workload as higher than did the pC-9 pilots, 
but again the differences were small. The ratings by the NASA 
observer and the LOFT instructor tend to favor the performance of 
the DC-9 crews, but again the differences were very slight. 


Considering the fact that the experience level in type was much 
higher in the DC-9 than the MD-88 crews (see Chapter II) , the 
picture is even less clear. There were no measures on which the 
MD— 88 crews were seen to perform in a manner superior to the DC— 9 
pilots, but the time-in- type factor cannot be ignored. One way 
to look at the data is to say that the automation made it 
possible for the MD-88 crews to compensate for their lack of 
experience in type. 


As a way of estimating the importance of time m type, taking the 
DC— 9 and MD— 88s separately, we correlated the months— m— type with 
a variety of performance measures: overall performance rating. 

Type 1, 2, and 3 errors, captains' and first officers' assessment 
of total workload, and captains' and first officers' assessment 
of frustration and physical demand. No significant correlations 
were found. Thus we cannot say that, at least within fleets, the 
experience level in type affects any of our performance measures, 
and by inference, the differences between DC-9 and MD-88 time-m- 
type figures may not be important. 


Another interpretation would ask why it is that the crews of a 
technologically superior aircraft could not do better, compared 
to those flying a 1965-era cockpit technology? Still another 
interpretation would be to say that crew experience we saw in 
this experiment was not an artificial experimental factor, but a 
representation of the "real world" of line flying? that for 
least a few years to come, at most airlines world-wide will 
continue to consist of a large group of pilots highly experienced 
in type in technologically unsophisticated aircraft, and a 
somewhat smaller group of less experienced (in type) pilots in 
advanced cockpits. Thus our samples represent the present-day 
"real world" of airlines with technologically mixed fleets, with 
the highest experience being by far in the less advanced 
cockpits . 


The results of this LOFT experiment provide both good and bad 
news to those who design, build, regulate, and operate modern 
aircraft, and reflect the typical dilemma faced by those who must 
provide training. On the positive side, despite pilot reports 
and concerns raised in a variety of previous research studies, 
the performance of crews in the automated aircraft was as a 
practical matter no worse than those in the corresponding 
traditional aircraft. On the negative side, despite the design 
goal of workload reduction on the automated aircraft, crews in 


52 



the MD-88 performed no better, and reported perceptions of higher 
workload than DC-9 pilots flying the same LOFT. 

As a general conclusion, it appears that the automated 
environment is neither better nor worse than the electro- 
mechanical: it is simply different. These differences will be 

emphasized in the volumes that follow. 

Pilots, through their training and efforts are adapting, as 
pilots always have. But in the current era of enlightened human 
factors awareness, the designs should be more "user friendly” to 
the pilots and not require so much adaptation. The human pilot 
can only adapt so much — and then an incident occurs and it is 
attributed to "pilot error” . 

The question for the research and development community remains: 
how do we provide the best pilot-aircraft interface? The 
question for the operational community remains, given the current 
aircraft designs, how do we provide the best training, 
procedures, and support necessarily to operate a new generation 
of equipment? 
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V. 


QUESTIONNAIRE RESULTS 


A. OVERVIEW 

Questionnaires were designed to elicit pilot opinions, experience 
level, and specific information and viewpoints. It was somewhat 
difficult to design a questionnaire that would be equally 
responsive to the needs and opinions of DC— 9 and MD— 88 crews. 

Most MD-88 pilots had flown traditional cockpit aircraft? but 
only eight of the 42 DC-9 pilots (all captains) had experience in 
advanced airliners (B-767/757) . In this study we do not take 
into account military or corporate aircraft the volunteers may 
have previously flown. 

The questionnaires were mailed to the volunteers in January 1990. 
The LOFT flights had been completed at the end of August 1989. 
Most of the questionnaires were returned by the end of the March; 
a few straggled in during April and May. They were designed with 
the goal that they could be filled out in one hour . Some 
respondents attached lengthy answers to some questions , often 
written on typewriters or word processors, indicating rather 
strong feelings about the topic. 

Questionnaire Components 

The questionnaire contained the following parts: 

1. Biographical information on flyincf experience at the host 
airline, total flying time, time m type, etc. These 
results are summarized in Chapter II- 

2. A 28-item Likert attitude scale, adapted from the one 
previously used by the author in his field studies on the 
MD-80 (Wiener, 1985) and the B-757 (Wiener, 1989). Some 
probes were identical to those in the previous studies, 
others were tailored to this study. An attempt was made to 
design probes that were equally sensible for the DC-9 and 
the MD-88 crews, however it is clear that many of the DC-9 
pilots had no experience with advanced technology aircraft, 
so their answers would be based on their general knowledge 
of automation, what they had seen and heard. 

3 . Information was sought on the number of various types of - 
instrument approaches (e.g. VOR, ILS, LOC) that had been 
made in the last year by the crews. Questions about 
Category III approaches and autolands are not appropriate 
for the DC-9? otherwise these questions applied equally to 
both types of aircraft. 

4 . Eight open-ended questions allowed the respondents . to 
express in their own words attitudes regarding various 
aspects of their flying experience. Open-ended questions 
gave the volunteers the opportunity to spell out in detail 
their opinions or experience in response to a variety of 
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issues (see Appendix 2 for a list). The final question was 
the most general. Volunteers were asked to relate anything 
about the human factors of their flying jobs that they 
wished to discuss. 


B. ATTITUDE SCALES 
Likert Scale Construction 

A Likert scale is a standard tool in attitude assessment. It is 
a form of "intensity scale ," whereby not only the direction but 
intensity of the response is measured. An item consists of a 
,, probe ,, , which is a positive or negative statement to which the 
respondent is asked his degree of agreement/disagreement. The 
response scale contains an odd number of possible responses, 
typically five or seven levels from strong agreement to strong 
disagreement , with a neutral value in the center, The center 
response is somewhat ambiguous: it can mean "no opinion", 

"undecided", or a truly neutral or centrist position on the 
probe. In this study, five response levels were employed: 
"strongly agree", "agree", "neither agree nor disagree", 
"disagree", and "strongly disagree". The response form is shown 
in Appendix 2. Note that in the histograms that follow, we have 
labeled the center response as "neutral" for brevity. This word 
did not appear in the questionnaire. 

Handling of Numeric Data 

The probes are referred to as Item 1 through Item 28, and data 
are displayed as histograms. The 28 histograms which follow 
simply report the percentage of responses to each probe at the 
five levels, by type of aircraft the respondent was currently 
assigned to. 

Numerical data from the questionnaires were entered into a 
computer-based file, and statistical analyses were performed. 

For each of the 28 attitude items (PI to P28) , two-way factorial 
2 -by - 2 analyses of variance were performed. The two dimensions 
were automation level (DC— 9 versus MD— 88) and seat (captain 
versus first officer) . 

In none of the 28 Likert probes was there a significant 
difference between the responses of the DC-9 and the MD-88 
pilots. Thus it appeared that one's current aircraft type, which 
we take as a surrogate for level of automation, did not affect 
one ' s attitude in a variety of questions on automation and other 
aspects of line flying, equipment, and safety. 

In only one of the contrasts of captain versus first officer was 
there a significant difference. This was in Item No. 27, 
"Automation capability enhances safety", the results of which are 
displayed in Figure V-l. It appears from the figure that the 
captains agreed with the probe more than the first officers. 
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One of the two-way interactions (automat ion-by-seat) was 
statistically significant. This was Item No. 7, "I use the 
automation mainly because my company wants me to.” These data 
are displayed in Figure V-2. The significant result indicates 
that the effect of automation and of aircraft seat did not 
combine linearly to produce the results. This interaction is 
difficult to interpret. The main source of the interaction seems 
to be in the "neutral" and "disagree" responses. Looking at 
these responses in the graph, one can see a disordinal 
interaction between aircraft and seat. More MD— 88 captains than 
first officers are neutral, and the opposite for DC-9 crews. 

When one examines the "disagree" response, the opposite is true. 



Figure V-l. A significant difference between captains and F/Os. 
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Figure V-2. A significant interaction 
between seats and aircraft type. 


Interpretation of 


Attitude Scales 


The reader is cautioned that the information in the attitude 
scales, as in the open-ended questions, is subjective opinion of 
the volunteer pilots. It is valuable information, both for 
researchers and persons in the aviation industry , attitude 
questionnaire data should be taken seriously. But likewise, it 
is important that any particular result not be over- interpreted 
by the reader, and that be clear that such data are not from 
essentially experimentally controlled conditions. 


The following seven pages contain in histogram form the results 
of the 28 Likert scale attitude questions. 


5 : 




PER CENT RESPONDING 


Flying today is more challenging 
than ever. 



LEGEND 

im MO-88 


1 1 DC-9 


STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 
AGREE . DISAGREE 

PILOT'S RESPONSE 



The DC-9/MD-88 automation works 
great in today's ATC environment. 

I LEGEND 


V/Z/A MO-88 
1— J DC-9 



It is important to me to fly the 
most modern plane in my 
company's fleet. 

I LEGEND 



STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 


STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 













PER CENT RESPONDING PER CENT RESPONDING 



PER CENT RESPONDING PER CENT RESPONDING 


I always know what mode 
the automation is in. 




In a highly automated plane, 
you run the risk of loss 
of basic flying skills. 











PER CENT RESPONDING ' PER CENT RESPONDING 


9. Automation frees me of much of the 
routine, mechanical parts of flying so I 
can concentrate on "managing" the flight. 



LEGEND 


STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 
AGREE . DISAGREE 

PILOT'S RESPONSE 



I look forward to more automation - 
the more the better. 

| LEGEND 



I have no trouble staying 
ahead of the plane". 



STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 


STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 













PER CENT RESPONDING PER CENT RESPONDING 



PER CENT RESPONDING PER CENT RESPONDING 


Automation does not reduce 
total workload. 



LEGEND 


STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 
AGREE . DISAGREE 

PILOT'S RESPONSE 


16. Flying the DC-9/MD-88 in congested 
terminal areas such as Washington and 
New York is not particularly difficult. 

— | LEGEND 



STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 
AGREE . DISAGREE 

PILOT'S RESPONSE 











PER CENT RESPONDING PER CENT RESPONDING 


17. Training for the DC-9/MD-88 
was as adequate as any training 
that I have had. 



LEGEND 


STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 
AGREE . DISAGREE 

PILOT'S RESPONSE 


9. I prefer the two-pilot cockpit 
to the three-pilot operation. 



LEGEND 


STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 
AGREE DISAGREE 


PER CENT RESPONDING PER CENT RESPONDING 














PER CENT RESPONDING 


I use automation mainly because 
it helps me get the job done. 



Our CRM training has been 
helpful to me. 


LEGEND 



STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 
AGREE . DISAGREE 

PILOT’S RESPONSE 



Sometimes I feel more like a 
button-pusher 1 ' than a pilot. 


There are still modes and features 
of the DCr9/MD-88 automation 
that I don't understand. 



STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 
AGREE . DISAGREE 

PILOT’S RESPONSE 

















PER CENT RESPONDING j PER CENT RESPONDING 


25. There is too much workload below 
10,000 feet and in the terminal areas. 

LEGEND 


V2 / A MD-aa 
1 I DC-9 



STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 
AGREE . DISAGREE 

PILOT'S RESPONSE 



PER CENT RESPONDING PER CENT RESPONDING 



28. Electronic flight instruments 
("glass cockpits") are a big 
advance in safety. 



STRONGLY AGREE NEUTRAL DISAGREE STRONGLY 








C. FREQUENCY OF USE - INSTRUMENT APPROACHES 

Volunteers were asked to Indicate in blanks on the questionnaire 
forms the number of times in the last year (1989) that they had 
used various instrument approach modes: autolands (MD-88 only) , 

Category III 115 (MD-88 only). Category II ILS, VOR, LOG and ADF. 
The data are displayed in graphic form on the pages that follow. 
No attempt was made to perform statistical analyses on the data, 
or to contrast DC-9 with MD-88 usage. These figures represent 
documentary data, for the purpose of reporting an estimate of the 
level of usage of various modes in the traditional and automated 
aircraft in daily line flying. 



Figure V— 3 . Number of autolands in the last year, MD— 88 only. 
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Figure V-5. Number of Category II approaches by aircraft type. 
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NUMBER OF APPROACHES 












Figure V-8. Number of ADF approaches by aircraft type. 
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D. OPEN-ENDED COMMENTARIES 
Data Handling of Open-Ended Questions 

Replies to the open-ended questions were individually read, 
analyzed, and classified by the author. Much of the analysis of 
these free text responses was subjective. Direct quotations were 
chosen to represent a variety of viewpoints. There was no 
attempt to make the number of quotations on any viewpoint 
represent the proportion in the database. They were selected for 
inclusion on the basis of representing a variety of opinions. 

The quotations are as close to verbatim as possible. The author 
performed "light editing” to make the quotations more clear, 
improve punctuation where needed, and put them into complete 
sentences. Where words are underlined, for emphasis, these were 
the choices of the respondent, not the author. In a few places 
editorial insertions were made for clarity, and these are 
delimited by the symbols < >. On several questions, results 

were tabulated and presented prior to the body of direct 
quotations. The two-digit number at the end of each quotation is 
for cross-referencing. 


Quest . 


Topic 


Page No. 


1 Errors pilots have made or seen 

2 Crew coordination 

3 Training 

4 ATC environment 

5 Workload 

6 Likes and dislikes of cockpit features 

7 Differences between MD-88 and DC-9 [1] 

8 Anything on human factors 


70 

81 

87 

92 

98 

103 

108 

115 


We ask the reader to keep in mind that in interpreting the open- 
ended replies, most of the MD-88 pilots were new to the aircraft. 
This may explain some of the negativity, as it takes longer to be 
"comfortable" (word the pilots typically use) in these aircraft 
than in traditional aircraft (Wiener, 1985) .. 


[1] Only answered by pilots who had flown both a traditional 
and a glass cockpit aircraft (MD-88 or 767/757) 
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1 . Describe in detail an error which you made, or observed, in 
the DC-9/MD-88 that could have led to an incident or violation. 
How could it have been avoided? (equipment design, training, 
crew coordination?) . Please describe specifically what occurred? 


Responses to this question are listed below. To the greatest 
extent possible, they are verbatim from the volunteers own 
words. In some cases minor editing was required. 


Due to the prevalence of reports of altitude deviations ( busts ) 
and other problems related to altitude acquisition and 
maintenance, the reports are divided into those that involve 
altitude, and all others. 


The two-digit number at the end of each report is a questionnaire 
serial number, included for cross-referencing. In those cases 
where the respondent indicated that he was no longer occupying 
the seat which he held at the time of his volunteering for the 
study, the new seat is included in brackets. 


The problem of "killing the capture” on the MD-88, which appears 
both in response to this question and to others, was first 
encountered by the first author in a previous field study, 
dealing with crew transition to the MD-80 (Wiener, 1985). The 
problem arises from the FMC logic when the pilots, intentionally 
or otherwise, change parameters while the aircraft is in the 
altitude capture mode, but has not completed the capture 
maneuver. A change in parameters, notably vertical speed or 
airspeed, may result in the aircraft departing from capture mode, 
and thereby failing to level off at the target altitude. 


MD-88 Reports - Altitude Deviations 

Climbing out of BHM we were cleared to FL 270. Before reaching 
altitude, we were cleared pilot* s discretion down to FL J40. We 
rearmed the flight guidance system to FL 240, thinking that the 
airplane had already captured the FL 270 altitude. Actually we 
had disarmed the alt hold mode and the aircraft continued to 
climb to 28,500 before ATC and we caught the error 
simultaneously. We should have actually watched the aircraft 
level off and not assumed anything. Automation tends to lead .to 
complacency which takes improved crew diligence and coordination 
to overcome. 09 

Possible altitude deviation and accident. Climbing out set 
altitude window and observed it arm. Only due to other traffic 
were cleared only to FL XXX. We got busy with routing changes, 
and looked up and saw that we were going through our assigned 
altitude. I then noticed that the altitude window had changed to 
a new altitude which was not selected. Most people use the 
altitude window as the place where you ("symbolically”) annotate 
your altitude. As a result of this incident and a few others, I 
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now write down on a piece of paper the altitude assigned. So in 
this case, instead of decreasing workload, the result was the 
opposite. Another example —— all the automatic warning systems 
that are supposed to be there to catch the errors by the pilots 
such as take-off warning horn. I do not trust them to do their 
job, i.e. back me, the human, who is liable to make errors. 11 

(L— 1011 F/0) 

The MD— 88 has a unique altitude bust feature built into it, with 
the autopilot engaged, autothrottles engaged, and VNAV engaged. 
When the pitch window of the FMA is showing VNAV/LVL and you push 
the SPD/SEL button, the pitch window goes to VERT/SPD, not 
ALT/HLD and generally a 100 fpm climb or descent will develop. 
Most of us are aware of the problem and don 1 t do that anymore , 
but design could prevent that. 12 


Captain moved the IAS wheel while A/C was capturing altitude. It 
canceled the altitude hold and went 200 feet low before I called 
out altitude. 13 


Level at FL 350, clearance to descend to FL 240 at 250 kts. JAS 
250 was selected to descend instead of VNAV . Aircraft pitched up 
to get 250 kts. We busted through 35,400 before disconnecting 
the autopilot and then descended properly. Why not go into alt. 
hold or no climb until speed is dialed in and selected? The 
descent on the MD-88 still gets many new pilots m trouble, 
particularly those coming off the 767. 20 


I failed to make a crossing restriction because I expected the 
aircraft (MD-88) to perform the descent. I had not programmed 
correctly and did not closely monitor because an F/A <f light 
attendant> offered us a meal. (There would be no opportunity to 
eat at the airport) . Double problem: 1) scheduling (no 

provision for eating) ; 2) inappropriate monitoring of automated 

flight. 21 


F/O climbing to 10,000 assigned altitude, VNAV and autothrottle 
on, autopilot off. He leveled at 10 without trimming nose down 
and immediately engaged the autopilot. As the A/P trim tried to 
trim nose down, the altitude increased a couple of feet, enough 
for the VNAV to sense above 10,000. This caused the autothrottle 
to advance to climb power, completely overpowering the ability of 
the slow autopilot trim to keep the nose down and hold altitude. 
The A/C pitched nose up and began a rapid climb. I caught it at 
10,200 with the VSI indicated 1200 + fpm. Avoid by trimming 
before engaging the autopilot. 22 


Captain* s leg. Holding instructions given by ATC and captain 
decided to copy holding instructions as well as F/O. Aircraft 
was in a descent and when holding altitude given, ^ nobody dialed 
it in and altitude bust occurred. Some has to "mind the store . 
If captain had been doing so or if he was hand flying the 
airplane, he would not have had the freedom to write instead of 
flying. 35 
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We were climbing out to cruise altitude and were given an 
intermediate altitude level off. We dialed in the new altitude. 
The plane was climbing rapidly, so the captain started to 
decrease the rate on the vertical window. This disarmed the 
altitude capture. I noticed it 200 f®et below level off 
altitude, and the autopilot had to be disconnected so we didn t 
fly through the assigned altitude. <See also report No. 13 
above, next report (62), and comment at the beginning of this 
seetion>. 58 (767 F/0) 


Several altitude excursions due to manipulating vertical speed 
wheel on FGS panel after the altitude capture, which kicks A/P 
out of capture mode. 62 


I feel that the biggest problem in the MD-88 is altitude 
excursions. I had three excursions and one f ail-to-capture . 
Leveled off at 13,000 accelerating, F/0 getting out an approach 
plate, and I was looking at radio panel when altitude alert went 
off. We were climbing through 13,300 when I disconnected and 
recovered. Second was in traffic pattern at BWX, downwind to 
base turn at 2000, and third was entering downwind leg at ATL. 
All three incidents traced to inadequate autopilot trim rate. 

85 (727 captain) 


MD-88 Reports - Other 

Arriving into ATL, a line of severe CB's were depicted on the ND 
as just west of the runways. A/C were landing to the east. I 
stated to the captain about 10 minutes from landing that I 
thought we should hold in the clear to the west of ATL until the 
CBs moved across the field to the east. X was concerned about 
landing with a tailwind if not windshear. With the CBs east of 
the field, they would not be a factor in approach or landing. 

The weather depiction on the ND was invaluable in forming my 
opinion. The captain, however, pressed on with the approach, and 
just prior to entering the weather the wind were reported as 280 
degrees at 13 kts. (exceeding our tailwind limit) • We executed a 
missed approach into a CB and tried to coordinate a turn back to 
the west in moderate turbulence. 5—10 minutes later the storms 
moved off the field and a very easy visual approach was made. 

More training in windshear avoidance involving geographical 
relationships of A/C, airport, and CB would help. The MD-88 
ND/weather feature is excellent in displaying realtime weather 
scenarios. 10 

On ILS final encountered severe turb and rain, executed a missed 
approach. Map failed due to no VOR update. We were told to hold 
at intersection, but couldn't find it due to map failure. (Notes 
intersection we were to hold at was off of an inoperative VOR) . 
Then autopilot began having trouble holding altitude (we are now 
unsure of where we are and had to hand fly pr monitor A/P very 
closely. Now both of us are trying to find where to hold, 
monitor the A/P, and talk to company and passengers. (A real 
mess) . Solution: one fly and one do the rest - and have maps 
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out:; communicate; inertials would help (no map loss) . 14 

Set up at night to do an ADF approach. The FMS was not set up 
for that approach, so I tried to build it myself. 
for a descent before the ADF for final approach. .Going to MDA 
early could have been disastrous. We need.IRUs < inertial 

reference units> and complete lists of _2Y ail ^ bl ® a P§ r S > ? c ^5 S i qSi 
each airfield in FMS. 15 <Note: starting in the fall 

all of the company* s MD-88/90 series aircraft will be delivered 
IRU equipped. > 

Landing at DCA, River Approach, F/O flying. As he called for 
be fore- landing C/L, I lowered gear, flaps, read C/L, then when I 
was able to look outside again he had flown past R/W 18. We 
could not get back to the runway, so we landed on 21 (with 
tower* s OK). I would avoid this happening again by: l) a good 

briefing? 2 I do not take my eyes off of runway during C/L (or 
any other time) . Of course a three-man crew would have avoided 
this problem, but as I indicated earlier <attitude scale>, I am 
neutral on that subject. 18 

We were cleared to hold at an intersection and the FMS depicted 
holding was similar, but not same as depicted on the hold 
depicted on the STAR. I let the FMS (LNAV) start the . holding 
pattern before I caught the difference. Luckily the initial turn 
was in the same direction. I should have been more careful - the 
copilot too. I was flying but he was engrossed m the FMS hold 
as he had never used LNAV for a hold. Never trust automation 
without checking it out first. 19 

We were approaching Austin from the east north east in absolutely 
clear and visibility unlimited weather at night. We were heading 
straiqht for the SPICE outer marker and were anticipating a 
visual approach to 13R. Both the F/0*s and my radios were tuned 
to the 13R ILS frequency and our courses were set properly. Both 
of our nav displays were in the map mode. In other words we were 
doing everything right - ILS armed to capture. The controller 
cleared us for the visual approach and asked us to keep the speed 
up and keep it in tight as much as possible for spacing on other 
landing traffic approaching from the northwest. I complied with 
his request and turned base leg inside of SPICE OM. I started my 
turn to final, looked out of my window during the turn and runway 
lights and VASI-L and rolled out on that runway. The tower 
controller advised us that it appeared to him that we had rolled 
out aligned with R/W 17. A quick glance at the instruments and I 
knew he was correct, and immediately rolled to the right and back 
to the left and was in the slot for 13R and landed without 
incident. The controller had just saved me. Both runways nave 
VASI-L but there are no approach lights to 17 , something we both 
should have picked up immediately, but didn*t. Also, there are 
usually some bright lights just south of and 1-2 miles west of 
the extended centerline of R/W 13R which illuminate a number of 
ball fields and some tennis courts. I usually look for them to 
assist me in finding the approach end of runway 13R. I don t 
recall seeing them that particular evening. I normally change my 
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nav display to full compass rose when intercepting the localizer 
whether operating IFR or VFR. I did not get around to doing that 
this evening - probably because I was using the map to turn 
inside of SPICE, This could have been avoided had I refused to 
cooperate with the controller's request and flown an approach 
right over the SPICE OM. I don't allow controllers to talk me 
into doing anything I am not comfortable with and really had no 
particular concerns with this request, I later called for our 
next ATC clearance and asked that controller to pass along to the 
tower controller my gratitude for his actions. The F/0 seemed a 
little surprised that I would do such a thing as admit to my 
error over the radio. I would like to think that either the F/0 
or I would have discovered the error before landing on the wrong 
runway, but since the controller was the first to catch it, 
thankfully we* 11 never know. 44 

I programmed FMS with one piece of information (a holding 
pattern) but did not execute it since it was for my own 
information only • Captain at a later point programmed something 
on his FMS. The Execute button illuminated on both FMSs due to 
my arming the system, and he hit the button. It inserted my 
holding pattern into the flight path even though we were not 
cleared and aircraft started a turn. Easy catch, but other goof- 
ups wouldn't be as easy. Better CRM <cockpit resource 
managements would have cured this one. 55 

Tuned wrong loc frequency for 8L in ATL. Did not identify the 

to con ditions - VFR daytime. Armed ILS and assumed 
autof light would capture localizer. I realized something wrong 
when A/C depiction on nav display showed us going through the 
localizer. This could have been avoided by: 1) have PNF verify 

the loc frequency; 2) identify the nav aid regardless of the 
weather? 3) ensure at least one pilot is in arc mode? 4) not 
relying on the autopilot to the place where you become 
complacent. 56 

On approach to BHM R/W 5 with both radios tuned to the ILS the 
F/0 was navigating in the map mode. We were cleared for the VFR 
approach and the F/O continued for the runway displayed on the 
ND, which could have put us 1/2 mile south of the actual R/W 5. 
When NAV 1 and 2 are tuned to the ILS, the FMS no longer 
accurately updates its position and goes to dead reckoning when 
ILS's are selected, and the switch <ND display mode> should be in 
arc or compass rose, not map! This is one of several such 
incidents. The AHARS should be replaced with IRUs as in the 
756/767. 59 (L-1011 F/0) 

fr 2 m fcts. at 11,000, then being cleared down to 8000. 
F/O began descent using IAS mode, selecting IAS 250. He did not 
notice that the trim rate was too slow to bring the A/C speed 
back to 250 below 10,000 until we could select alt hold and 
extend speed brakes. to comply with speed restriction. I was 
■ racted by complying with route changes issued along with 
altitude change - direct intercept input to FMS, and frequency 
change at the same time. Training to complete airspeed change 
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prior to selecting lower altitude might have helped. 60 


The captain loaded the flight plan into the FMS for a flight from 
HOU to CVG. The fix just prior to IIU (Louisville) was MYS 
(Mystic), but the captain saw the unfamiliar MYS on the flight 
plan and entered the familiar MSY (New OrieansTyinto the . 

computer. I caught the error and pointed it out to the captain. 

83 


DC— 9 Reports - Altitude Deviations 

A very eager F/O took the initiative to set in assigned altitudes 
and even thought he was PF, and as PNF our procedures call for me 
to acknowledge on radio, then set the alt alerter. After several 
instances of this F/O quickly setting it himself, we were set up 
for me to assume he would set it the next time. He was busy and 
did not and we busted an altitude. These new F/Os should be told 
in training that their performance is not predicated on eagerly 
doing "extra” tasks. 7 


The most common error is the failure to level off and maintain 
proper altitude. Too many times during the last 1090 feet of 
climb or descent distractions such as "heads down" in the charts, 
frequency changes, etc. cause altitude deviations. There always 
seems to be a frequency change or runway assignment given about 
level off time. 8 


Altitude bust. Heard the AAD in my subconscious, however went 
1000 feet below my assigned altitude. Copilot could have caught 
it. It should have a voice <alert> that says "Altitude, 
Altitude". 36 


While cruising at 31,000 the autopilot was inadvertently turned 
off or kicked off by itself and the A/C be^an a slow descent. We 
noticed it at about 250 feet below our assigned altitude. 48 


After takeoff the F/O hand flew the aircraft to 10,000 feet, 
leveled off and engaged the autopilot. There was turbulence, and 
scattered buildups in the area. We also received a frequency 
change while attempting to avoid the weather. I observed the. 
airplane to be descending toward 9,500 feet and pointed this out. 
This is not an uncommon error. It could be prevented by an 
"altitude capture" mode. On the DC-9, when the A/P is engaged, 
the vertical speed existing at time of engagement is maintained. 
67 

Altitude excursion seems to be the problem on the DC-9. Fairly 
frequently the PNF prevents an altitude excursion and twice I had 
been involved in missing an altitude, once in climb and once in 
descent. The scenario seems to occur when the PF and PNF are 
distracted by separate events/problems and both lose altitude 
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awareness. The autopilot is engaged and seems inadequate to 
capture the pilots* attention. The 1000 foot prior <to assigned 
altitude> call helps, but is not always made. Perhaps the PF 
should fly the A/C after the first AAD alert until level off? no 
looking at charts, talking on separate frequency from PNF, 
talking to flight attendants on the interphone, adjusting radar 
— some of the common distractions. 82 

Altitude bust descending on initial phase of descent during 
approach briefing. It was caught after we were 300 feet low. 
Better crew coordination and "flying the plane" would have 
helped. I suppose auto altitude capture would have caught it, 
but this is no substitute for staying heads up. 


DC— 9 Reports - Other 

During one of my first months on the line I aborted a takeoff 
just prior to V— 1. The reason I aborted was because of an 
autothrust disengage light illuminating. This is a system rarely 
used on a DC-9 and turned out to be a nuisance light. Looking 
back it was an unnecessary abort, however at the time I was 
concentrating on the takeoff and waiting to hear the first 
officer call "V-l". The light in question is red and to me, red 
is bad. I gained view of the red light from my peripheral vision 
and executed an abort. I feel this is poor design. It may be 
necessary for the light to be red and the system is rendered 
inoperative on the ground by the ground shift, however the nose 
strut begins to decompress near V— 1 and the aircraft goes to the 
flight mode thus enabling the system on the ground . Granted this 
was probably just a fluke, but it caused a high speed abort. 2 

F/O didn't communicate during arrival STAR. He twice dialed in 
the wrong course or got distracted with speed control and didn t 
update or fly published course. Since I'm a fairly new captain 
(eight months), I've watched things very closely to make sure 
items are corrected before they cause a problem, and have had 
good luck so far. 3 

Victor airway on arrival had large left turn to station (about 
120 degrees), clearance to cross intersection at 10,000, F/0 
read intersection at 67 DME when it was really 37. We had 
confirmed outbound course, and intersection at 10,000, but not 
the distance. If I had not questioned the descent point we would 
have busted the altitude at the intersection. A glass cockpit 
would have avoided this: distance to waypoint readout, visual 

display of waypoint, green arc projection of point at which 
target altitude would be obtained>, and FMS display readout. 5 

This occurs almost every flight into DFW or ATL when it is the 
F/O's leg. I try to get a gate and Approach or Tower will call 
and I'll miss their call. The company wants us to call about 10 
minutes out in DFW and closer at ATL. I know how to avoid this: 
by calling for a gate when we are on the ground. I used to do 
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this, but I was told, "Earlier.” DFW will usually say "call when 
on the ground." This <early call> was one of the things the 
company said it would not require when we first started flying 
the DC-9 . 6 

Captain* s leg on an early departure after a very short layover. 

No other traffic into ATL and 3 previous controllers told us to 
keep speed up. Neither of us noticed that we were indicating 340 
knots at 8000 feet, until Approach called us and asked our speed. 

X was new to the F/0 seat, and failed to notice that the captain 
didn’t slow at 10,000. I was studying the approach. 24 [767 

F/0] 

<Note: the following incident included on the questionnaire 

took place on an Air Force DC-9 (C-9) , which is outside of 
the scope of this study. It is included m this report 
simply because it is an interesting, very critical 
incident . > 

While in the Air Force, on the eighth leg of an eight-leg day I 
had a new copilot in the right seat. He was tired and 
overloaded, so he was sitting back sprawled out with his left 
hand resting on the flap/slat handle (fairly common position). 

We finished the checklist and took off on the roll. As I called 
for gear up, his left hand stayed on the flap/slat handle and he 
brought them up. Fortunately the DC-9*s flaps and slats move 
slowly, and I reached over and extended them. The DC-9 family 
has a problem in its design when the copilot is seated 
comfortably usually rests his hand on the lever. In my opinion 
the design lets people get comfortable in a position that when 
they are tired they move the lever that is most convenient. 

Since then, when I am tired, I remind myself of what can happen. 
28 

We were on a standard arrival to IAH and were supposed to 
intercept a radial inbound to a VOR from an intersection. I, the 
F/0, set the inbound radial in my course select window and 
simultaneously told the captain what the inbound course was. He 
was still tracking the outbound radial from the last VOR. He 
acknowledged me, but did not set the new radial in his window. I 
failed to notice after about two minutes that we had passed the 
intersection and the captain was still tracking the outbound 
radial. I told him that we needed to turn back to intercept, 
just as ATC called us and asked if we had intercepted the inbound 
course yet. If the captain had put the new course in his window, 
or monitored my PDI, he would have noticed that we had passed the 
intersection. If I had watched him closer, we would not have 
flown past it so far. 29 

Waiting to cross an active runway, Tower told us, "proceed 
straight ahead across runway at B, aircraft taking the active 
will hold for you, expedite across, contract Ground <121> .7 when 
clear." The captain pushed up the power, and I looked in my Jeps 
to see the Ground frequency. Well, Taxiway B split and we were 
expediting straight. The captain kept looking left at the 
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aircraft holding for T/0, all lights on, and my initial call to 
stop was not heeded, and I ended up skidding us to a halt before 
we taxied onto the grass. We ended up less than five feet (more 
like three feet) from the grass. A happy ending, barely. How to 
avoid: 1) always look where you're ^oing? 2) crew coordination 

_ listen to your crew? 3]| communication ~ better terminology 
from the tower (no "straight across") . 32 

I was recently cleared HMV, FLM2 arrival to CVG. In error, I 
flew HMV direct FLM, CVG. Listening closer would have prevented 
this. However; if the DC-9 had a "glass cockpit", the proper 
routing would probably have been entered into the FMC , which 
would have prevented this kind of error. 39 

Last leg of a long day - dark, tired, failed to notice on flight 
plan that we were tanker ing fuel to Daytona Beach. ^ 

Daytona Beach was (at that time) a cutback runway, but didn't put 
two and two together until I was half way there. . I looked at 
flight plan a saw a "do not tanker" message. I finally realized 
that I had missed some information. I checked with flight 
control and found they had erred by dispatching aircraft with 
extra fuel. Checked as to runway length vs. our weight (we were 
close to max landing weight) . Flight control provided numbers as 
the book in our cockpit was so conservative that I would have 
been too heavy to land with zero fuel on board! Once we began 
thinking like pilots and not like bean counters I realized that 
we would have no trouble landing a DC-9 on the runway available. 
However, to cover ourselves we got all the exact figures and 
landed. Landing was uneventful, except I made the landing 
although it was F/O's leg. He was glad I did. This made me 
realize I could have pushed his limits if I had told him go ahead 
and land. So it worked out, but I check the figures now every 
time, tired or not. 40 


In the DC-9, the crew inadvertently timed the Falmouth VOR 
(117.0) instead of CVG VOR (117.3) and began tracking inbound. 

It occurred during a time of heavy workload during descent. A 
glass cockpit display and FMS would probably have prevented this 
event. A third cockpit member to handle ATIS, gate assignment, 
cabin PA's and checklist reading, might have also prevented this 
occurrence . 41 


This error occurred at Hopkins International Airport in 
Cleveland. We were parked at Gate B-17 and pushed back at a yery 
slow traffic period. We were the only aircraft moving on the 
field. The active runways for takeoff were 23 R/L. After we 
pushed back the captain started the first engine and left the 
second one for me to start while he taxied. The time it took to 
get to the runway was minimal. I coordinated ground clearance 
and takeoff clearance on the radio. I also accomplished the 
"after engine start", "taxi", "delayed engine start", and "before 
takeoff" checklists. It was my leg to fly. B Y the , 
finished all of this and got my head out of the cockpit the 
captain stood up the throttles and gave me the aircraft for 
takeoff roll . As soon as I started to advance throttles 
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Cleveland Tower told us to abort takeoff because we were on 
runway 28 (of which 6000 feet remained) . The Tower seemed ready 
for this mistake - I presume it had happened before. If you look 
at the runway layoff there's concrete everywhere. 46 


Dialed in wrong course for LOC/ILS approach and unable to get 
flight director and autopilot to fly the same approach. F/O and 
myself visually cross-checked each other to make sure we had the 
same course set in. 49 

1) Using autopilot turn knob to correct to a new radial, thought 
knob was in detent, but we were actually in a shallow turn. 2) 
Holding at marker awaiting ILS approach, strong winds, no DME, 
cleared for app and let down from 3000 to 1800. Asked to make 
360 degree turn for spacing, then cleared for approach. A lot 
closer to marker than we thought, resulting in large angle of 
intercept inside marker. Times like that would be nice to have 
DME or FMS . 52 


Failed to set brakes on engine start. A/C rolled toward 
terminal. Would be avoided by stricter adherence to checklists 
and crew coordination. 61 

Copilot landed in ATL with 5-kt. tailwind and rain. Landed in 
landing zone on a speed runway, 8L. I told him, ''you reverse and 
I'll brake (light rain).” Went to make high speed turn off and 
the aircraft started sliding sideways. I got in straight and 
stopped it on the high speed. Later I thought I should have gone 
to the end of the runway. Talked to a 767 captain that landed 
behind us and they actually had a 15-kt. tailwind at 20 feet. He 
called the Tower and told them to turn the airport around before 
someone got hurt. When I went back out the airport was turned 
around. 63 


On a back course ILS into Columbus, Georgia, at night, IFR, 
weather at minimums. As a new F/0 I forgot to put the proper 
course in the PDI prior to the approach. It was still set on the 
previous direct heading to the Colu mb us VOR. I was paired with a 
captain in his first month. When BL mode on the flight director 
was selected, steering was given to the course in my PDI instead 
of the correct back course. The captain was flying and followed 
the flight director. There wasn't any ADF available to back up 
the approach, so he descended on a course 50 decrees off of the 
proper one, centering the flight director steering. It was some 
time later that the discrepancy was noted. Had there been 
terrain or traffic and the error not detected, an accident or 
incident could have resulted. This could be avoided by approach 
checklist emphasizing placing the proper course in the PDI. 69 

Misread the outbound heading on a jet airway where two jet routes 
converged over a fix. The wrong heading was taken. 72 

Autopilot engaged, instructions to slow below clean speed. When 
we lowered slats autopilot did not retrim rapidly enough and A/C 
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lost 100 feet with continued descent. Deactivated autopilot and 
recovered manually. 86 


We were filed over two VORs and then out a specific radial from 
the second VOR. The two VORs are less than 30 miles apart, and 
differ by only 1 digit <.2 mhz.> (116.7 vs. 116.5) . The captain 
was flying and had switched my nav radio to a separate VOR at the 
destination field for another reference. We started out the 
radial from the first rather than the second VORas filed. 
Pertinent factors included a short (pushed) flight, the close 
proximity of the two VORs, and the similarity m frequencies. I 
would recommend that if a crew member is going to change the 
other crew member's nav radio, that he should tell him. I would 
also recommend that VOR stations so close should differ in 
frequency by more than one digit. If this is not possible, then 
X would recommend that the first digit <1 mhz.> be varied, ©*^g» 
115.7 vs. 116.7, instead of 116.5 vs. 116.7. <No serial number 
this report submitted after database was completed> 
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2 . 


What: can you say about crew coordination on the DC-9/MD-88? 
What areas of crew coordination/ communication need 
improvement? 


MD-88 Crews 


With the many options available as to how you want to use the 
different gadgets, it's important to tell the other guy what 
method you are using and what you are planning to do- F/0 9 


I like the policy of "my leg I'll do the FMS, your leg you do the 
FMS." With the autopilot off, then the PNF programs the FMS/FGS. 
However, this policy is only moderately followed. If I am flying 
and the captain programs the FMS, I am not always positive what 
mode of operation exists (i.e. crossing speeds and restrictions, 
etc-). If the captain updates something for me, I appreciate it 
if he verbally tells me what he did, at the least. F/O 10 


MD-88 crew coordination - the pilots are usually too busy if 
anything comes up from the cabin during climb and descent. Any 
call in a terminal area can be a real distraction or possible 
problem. We also have a hard time building crew rapport due to 
switching flight attendants so much. We need to improve 
communication between cabin and cockpit crew. Adding much more 
information to the required briefing is not the answer. Already 
the captains are spending time <with flight attendants> that 
should be used to help in the preparation of the cockpit, to the 
point where the F/O has to do everything if you are to leave on 
time. You need to keep the entire crew together and have a 
briefing when you check in for the rotation. Another factor is 
that some captains who come off three-man crews don't realize 
that you have to operate as eguals when it comes to workload. 
During emergencies crew coordination is imperative. < You 
constantly on the edge of the abyss of task saturation in a busy 
terminal area. In most cases the F/O cannot go back to check on 
the problem, and must take the word of a person who may not have 
the training to give a good assessment of the nature of the 
problem. It's very easy to have both <pilots> working on the 
problem and not have one guy maintaining aircraft control. F/O 
11 


I think it's improving, but there is significant room for more 
improvement. Sometimes I (as F/O) feel a real need to drag game 
plans (for abnormals) out of the captain. He should be telling, 
not me asking. Also many captains have their own set of 
procedures which tend to impede communication until you have 
customized yourself to him. Sometimes communication can be 
impeded by concern for FAA and company procedures. More concern 
for covering your rear than the big picture. F/O 14 

Coordination is good, CRM training is making it better. What is 
good for the goose is good for the gander. Captains are not God. 
If an F/O must do certain duties a particular way, then all crew 
members should do it the same way. I see too many captains who 
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believe "do as I say and not as I do." They will enhance 
communication, respect, and a better cockpit working environment 
if they adhere to procedures. F/O 15 

I think the captain sets the pace. I try to maintain an 
environment that is open and free to communicate. Judging 
(albeit subjectively) from the results of recent trips, it 
appears than open communications pay off in smooth and hassle- 
free trips. F/O 17 

Coordination is good. But there is too much talking (climb 

power, VNAV, flaps up, slats up, bank angle 25, etc. ) . 

some of the requirements that must be said out loud. Capt. 20 

Crew coordination is this aircraft depends on each crew member 
advising the other continuously of what they're doing. It is so 
easv to say "just do it" yourself because the other person is 
busy on the radio, etc. I don't always verbalize all my actions. 
As captain, I find that I can't watch all that the F/O does 
because of my own workload. Capt. 21 

I've seen too many occurrences of both sets of eyes glued on the 
FMS computer, when one set should have been monitoring the flight 
and airspace. F/O 42 

Crew coordination on the 2 -man crew is excellent -- far better 
than the 3 -man crew. There is less coordinating to do. As far 
as the rest of the crew - flight attendants - are concerned, 
elimination of this ridiculous "sterile cockpit" rule would be a 
big step forward. We often hear the flight attendants talking to 
each other over the interphone system about some problem they 
have and they are trying to decide whether or not they should 
call us to advise for fear that we will get angry with them for 
violating the sterile cockpit. Capt. 44 

Coordination is normally good, however, when one pilot is 
occupied with one duty (e.g. making a PA to passengers , obtaining 
ATIS on arrival, talking to company ramp to find out gate) the 
other pilot is unmonitored. Thus if he initiates an action or 
copies a clearance incorrectly the "out of the loop" pilot will 
never catch it. All the automation in the world won t catch 
that. Main area of crew coordination that needs improvement is 
one pilot verbalizing his future intentions or plans to the other 
so any errors in the plan can be caught quickly. F/O 55 

Overall I feel that coordination is good on the MD-88. I feel 
that coordination is a lot better when both pilots have been on 
the 88 for a while and fully understand the full capabilities and 
limitations of the FMS/autopilot. F/O 56 

Greater standardization is needed. The complexity of the 
autoflight system means an endless variety of techniques are 
used. Pilot disagreement on methods used is a constant problem. 
CBM doesn't really address the problems we have. Capt. 57 
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You need to assign jobs so both don't get tangled up trying to do 
different things on the FMC. F/O 58 


Havincr flown both the DC— 9 and the MD— 88, I realize a great deal 
more communications is necessary on the MD-88. More attention 
has to be paid to the details of operating this aircraft. There 
are too many ways this aircraft can turn around and bite you. 
Never alienate your helps he may not be there when you need him. 
On the DC-9 you could be a one-man show if it were necessary. 
Capt. 59 

X think crew coordination in the MD— 88 is good. Following 
company procedures , common sense , and . common courtesy , when _ 
combined allow for good coordination in the cockpit. Bringing 
coordination with the flight attendants m better focus by better 
briefing and better understanding of what services are being 
offered on a particular flight could stand improvement. Capt. 60 


Too much talking is required when hand flying, since 
manipulation of the nav and FGS panel is supposed to be done by 

PNF. F/0 62 


Briefings are important, particularly in bad weather. You need 
to communicate what automation you intend to use, and when you go 
to manual control. Further, when ATC changes a route or 
instruction, the PF should command how to reset the computer, or 
if to use it at all. Capt. 77 


DC-9 Crews 

F/Os need to talk more about what they are going to do next, i.e. 
"...at 14 DME I sun descending to 10,000 feet and turning right to 
296 degrees outbound." They are slow and tend not to communicate 
their intentions as much as I would like. Capt. 3 

On the whole crew coordination is excellent. If there should be 
any areas of improvement, it would be the standardization of 
procedures and information in line flying; and more emphasis on 
the F/Os input being desirable, even required as part of the 
communication loop. Capt. 4 

There has been great improvement on the DC-9 in the past Y5 ar ^2 
crew coordination. The CRM program has put in place the idea of 
the open cockpit. Capt. 5 

Crew coordination is OK, but not great, but OK. I don't like 
having the PNF moving the gear and flaps for me. Some times he 
doesn't hear me or sometimes I want them now and not later. If I 
say what I'm going to do and then do it, it seems OK to me. To 
retract the gear for the F/0, I have to reach all the way across 
the throttle quadrant and block <the view of> the instruments as 
a critical time. Capt. 6 
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The DC-9 is very straightforward with little/no automation, so 
crew coordination goes to basics. • .more a matter of personalities 
than equipment— related tasks. These new/ inexperienced F/Os 
should have it stressed to them that extra eagerness is not 
necessary to prove their worth ( see above) <See comment No . 7 , 
Question X>. The importance of PF/PNF duty assignments is no 
less in this cockpit than in the 757 • More is not better for an 
eager F/0. Capt. 7 

Generally DC-9 crew coordination/ interaction is excellent. 
Although I was a skeptic at first, the use of earpieces/headsets 
has greatly improved crew— crew and crew— ground communications, 
but reducing the "was that for us? H by about 90%. Capt. 8 

Crew coordination on the DC— 9 is generally pretty good because 
the cockpit is small and it's easy to monitor each other. 

Company procedures are adequate . Each captain needs to 
specifically brief each F/0 he flies with on his individual 
techniques. F/0 29 

Generally speaking crew coordination is satisfactory . This is 
one area that could be improved. The PNF should not change the 
nav radio frequencies without notifying the PF . The PF , whether 
he is captain or F/O, should be in charge of the nav radios. F/0 

30 

MD-88 crew coordination is more critical due to the higher 
workload than on the DC-9. Trying to hand fly an MD-88 while 
telling the other crew members how to program the DFGC/FMS is 
particularly labor intensive and clearly diverts both crew 
members' attention away from maintaining a safe lookout. F/O 33 
[F/O MD-88] 

I think it is basically good. Improvement could be made though, 
by each crew member flying verbally informing the . PNF of his 
actions. This should be done on a continuing basis. I am 
referring to such things as heading being turned for navigation, 
altitudes being vacated, crossing restrictions which are being 
flown. This is not something that should be put on a required 
checklist, but rather an attitude and standard operating 
philosophy . Capt . 3 9 

I have not had a single incident or problem on the DC-9 (since I 
began flying captain) . DC-9 operation lends itself to good crew 
coordination. If I have a guy who might be new, or a little 
weak, I slow things down and help him keep up with the aircraft. 
F/Os all try hard to please the captain. Actually they tend to 
be too hard on themselves, i.e. get down on themselves if landing 
isn't perfect in every detail. The biggest problem in crew work 
is with flight attendants. We sometimes change girls five times 
a day. This makes it tough to even know who is back there and if 
they know what to do in case of emergency. I feel same crew of 
F/As should fly the whole rotation (as American does) . This way 
you can get around personalities and not be put off by a possible 
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They 


bad first, impression. Note: 99% of F/As are outstanding, 

do a very hard job extremely well. Capt. 40 


Our present approach to CRM seems to be leading to more 
confrontation in the cockpit, between me and the copilots. I 
welcome suggestions and constructive criticism m the cockpit, 
but I see more of a challenge to my authority. At times the 
cooperation and assistance of my copilots seems to be less than 
prior to the days of CRM. The captain's authority as final 
decision maker must not be diminished. Especially in the case of 
new hire F/Os I have seen an increase in insubordination. Again, 
I welcome an open flow of cockpit constructive criticism in both 
directions. But what I am seeing is more of a this is the way 
we used to do it in the Air Force" mentality and less of a 
willingness to conform to the company's way of doing things. 

Crew members are encouraged in CRM to draw upon their previous 
experiences. However, this must be kept in proper perspective. 
The captain still must have the authority to "set the tone* in 
the cockpit, and expect the F/0 to carry out his assigned duties 
without having to give a long dissertation justifying each 
decision. Perhaps part of the problem is that our company is 
hiding some older pilots with higher experience levels resulting 
in a captain/copilot team with both pilots closer to the same age 
and experience level. A more likely explanation, however, is 
starting to hire more assertive types. The "B scale issue no 
doubt adds to friction in the cockpit. Capt. 41 


Overall coordination is excellent. The DC-9 is so simple and its 
systems are so well organized. Each crew member functions well 
with the company's current guidelines. Occasionally I work with 
captains who do not comply with certain operational procedures, 
but these have been few. Usually these same captains are the 
ones who don't communicate as well as they should. But I adjust. 
F/O 46 


Crew coordination is generally very good on the DC-9, especially 
with the new captains. When procedures are accomplished m the 
"flow" or order like we were trained to do them, crew 
coordination is good. Some of the older captains that have been 
on the DC-9 for years and still somewhat set in their old ways , 
which can cause a breakdown in crew coordination and 
communication. F/0 48 


Things are improving after exposure to CRM. We need to pay more 
attention to details of what person (PF or PNF talking 9 n _ 
radio) is doing. In good weather we tend to get lax and drift 
off into other thoughts rather than the apparent uneventful job 
at hand. Capt. 49 

Crew coordination is good on the DC-9 . You have to work together 
on a two— man crew, especially in bad weather or an abnormal 
situation. Sometimes there is much too much chatter when you 
taxi, especially at the big airports. Many times copilots try to 
read the checklist when I am busy figuring out where we are 
going. I just tell them to hold on a second. Training pounds 
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into them the checklist and the dual response. That is great in 
the simulator, but in the real world you have to teach the 
copilot how you want to do it. Capt. 63 

Our recent changes have been an improvement. The sterile cockpit 
concept has some merit, but it is a classic case of "them that no 
playa the game maka the rules". The captain is responsible for 
safely completing the flight and having a relaxed cockpit 
enviro nmen t has been one of the reasons our company has had the 
best safety record of all airlines in the past 35 years. Capt. 

67 

X don't have any concerns in this area. I think the operation is 
smooth and safe. Dual head com radios would help. Also the 
ability to receive messages (ATIS, and eventually ATC 
instructions) via datalink would allow both pilots to monitor ATC 
at all times. Capt. 74 

Crew coordination overall is very good and improving, 
particularly with the cabin crew. The biggest difficulty is 
communicating with the F/As. Some captains will not let me talk 
to the F/A when I ask to in a low workload time frame. 95% of 
the trips, by the end of the first day, I know what a captain 
expects and feel we work together well. By the end of the first 
trip I feel like there is a very good foundation for working 
together. It's the few guys who 1) insist on doing just about 
everything their own way? 2) feel threatened by any suggestion, 
question or initiative, that cause the problems. The impact 
seems to be that I, personally and together as a crew, .see 
performance drop dramatically • Xt frustrates me that m those 
situations I try harder to be even better and usually perform 
worse . I feel sure it is because the captain and I are not 
supporting each other. CRH sounds great during training, but 
it's not reaching these guys and I have yet to find anyone who 
has been able to effectively deal with these individuals. Their 
personalities which make them difficult to get along with also 
makes them CRM— ill iterate. F/0 82 

In general coordination is very good. The only time I experience 
much of a problem is when the captain "mumbles", or repeatedly 
does not speak loudly enough to be heard and/or understood (due 
to cockpit noise, radios, headsets, etc.) Eventually you get 
tired to asking him to repeat himself and just assume what you 
thought he said. F/0 90 
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3 . 


What did you think of your training for the rc-9/m-88? 

What topics should receive more/ less emphasis. Any comments 
on training aids and devices that were used, or needed. 


MD— 88 Crews 

TABLE V— 1. Listed below are replies made by two or more 
respondents (MD-88) . 

Need live instructors 

Systems training too shallow 

Long Beach training inadequate, used 

MD-82 materials with MD-88 differences 
Need more hands-on training devices 
Should have more FMS in ground school 
Course is too intense, concentrated 
CPT was excellent device 
Training devices were out of date 
Need ground school instructors with 
line experience 


There was a lack of personal instruction* The whole ground 
training program was done by video tape and because the plane is 
so new there were many questions and no experienced instructors 
with answers. The video tapes were good, but I believe an 
instructor should teach systems and be able to lead i n ~depth 4 -v,^ 
discussions on each system and their quirks. There is always h 
way a system is designed to work on paper and then how it 
actually works in real life. 9 

I was trained in Long Beach <Douglas>. Systems needed more depth 
and the program should have been MD-88, not MD-82 with 
differences training. Systems on 727 S/0 were taught in adequate 
depth and this is how we should be taught. Interactive computers 
are very good. We should head in that direction. > But do riot 
eliminate the teacher/ student classes where questions can be 
asked. 10 


I thought MD-88 training was great. The only room for 
improvement would be to have a flight training instructor or a 
better trained ground training instructor to run the > 

FMS/autof light and system CPTs. The ground training instructors 
just can't answer line— related questions. 12 


There should be more emphasis on systems and how they work. Much 
more emphasis on the things that can bite you: not capturing 

altitude; how you can <inadvertently> negate captures; map 
failures ; going into D/R and drifting; when to turn it 
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<automation> off because your are overloaded. 14 


Training in both aircraft was very good. In the full-time F/D 
aircraft. I feel that it is important to fly without the F/D 
(autopilot on or off) so that the pilot can become aware of the 
pitch attitudes required for various regimes of flight. This 
pertains to the first few days in the simulator before too many 
abnormals are introduced. 16 [Capt. DC-9] 


I feel that the training was excellent on all aspects of the 
plane except the FMS. I was expected to learn the FMS on the 
line and in fact that's where I learned the system. This 
definitely needs more emphasis. An FMS trainer was not available 
when I went through training and it is needed. 19 

Good training, but we need more emphasis on MCP, and less on ram 
air etc. Best training I have received at our airline. 20 

Not enough time . Full day in class . plus necessary study time for 
very long days. After a week at this pace, I . found myself 
saturated and my learning curve for new material peaked. X 
really felt pressured because it was also my initial captain 
upgrade. 21 

Training was excellent except for one thing. DC-9 to MD-88 
conversions were given only four legs as an IOE. This is 
inadequate since the 88 presents a radically different approach 
toward flying. 35 

I was pleased with the concept of the MD— 88 ground school - slide 
tape presentation. I do think, however, that in the future the 
person who narrates the tape should be a pilot. I found myself 
becoming aggravated at the reader of the text for several reasons 
that had an overall negative effect on my learning. It was 
obvious that he did not have a flying background because he 
either mispronounced words or put accent on wrong syllable. 
example, engine pressure ratio — we call it "E— per" , he referred 
to it as "E.P.R." As far as the training aids, I found them to 
be the best yet. As for the simulator, I think it is an absolute 
waste of time to sit in a simulator and before each period give 
the instructor a flight attendant briefing. I feel the same way 
about a departure briefing. Fine, if you have an unusual 
departure, but otherwise we all know who is going to do what, 
when, and how. Let's fly the simulator. We can talk during .. 
briefing and debriefing. Like a fellow pilot said, "We are 
sitting in Atlanta in the middle of the summer with OAT of 95 
degrees. If the F/O is so stupid that I have to tell him we 11 
not have to use engine anti— ice today, he doesn't belong in that 
cockpit. 44 

Training was basically good. I had real problems with the 
autoflight system. It didn't make sense to me. I kept thinking 
of the 757/767 system. A pitch wheel for vertical speed only. 
After training I went back to the DC— 9 for two months. On m Y 
first MD-88 approach, into DCA, my F/0 asked me if I was a little 
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high. Then Approach Control came in with, ”Do you think you can 
make it?” We made a 360 degree turn. I was at least 10 miles 
behind the aircraft, trying to use the autoflight system. Too 
much time between training and flying the aircraft. 59 [L ion 
F/O] 

There are some areas of the FMS that after a year are still 
ooorlY understood or uncomfortable to use. Discussion with other 
MD-88 pilots reveals same feelings. HLD and DES pages seem to be 
particularly troublesome. 71 

It was good overall, but to much drills on V— 1 cuts, missed 
approach, etc. This is necessary to meet the FAA criteria which 
are totally unrealistic. We are losing sight of the big picture. 
If x lose an engine, I want speed and altitude and to get back 
down, but I am not going to fly +/- 5 kts. and +/- 50 feet as in 
simulator drills. More time should be spend on real world 
problems rather than imaginary so-called critical situations. 

Have we ever lost an airplane due to losing an engine at V-l. I 
doubt it. 77 [Capt. 767] 


DC-9 Crews 


TABLE V— 2 . Listed below are replies made by two or more 
respondents (DC-9) . 

Need better CPT 

More systems training t , , 

Better incorporation of CRM into training 
More training on Jeps 
Simulators and CPTs not reliable, 
out of date # 

Too much emphasis on non-precision approaches 
Need instruction on ops specs, FARs, AIM 


I feel that the DC-9 school is a good school and always has been. 
One area that could be addressed more is performance. I can t 
comment on the training aids as they have been completely redone 
since my training. I have heard favorable comments about the new 
ones. 2 

The DC-9 training (ground) was excellent, but only due to the 
instructors. Flight training initially was not on company 
eguipment, creating concern and some confusion for first-time 
checkouts. Line training was adequate* Recurrent training has 
too many complex systems on the same day due to (I am told) FAA 
requirements for so much time to arbitrary subjects. 4 

Early training (sim 1-3) was poor. Training in a simulator with 


89 



entirely different cockpit from our DC-9s is poor, especially for 
two pilots new to the DC-9, e.g. HSI vs. no HSI. Obvious lack of 
ATC experience of my F/O partner (six months as 727 S/0) greatly 
impacted my training as I spent a great deal of time carrying his 
ATC load and/or actually training him on the spot on how to be an 
F/O in ATC environment. Half-day on introduction to Jeps was 
obviously inadequate, especially from a non-pilot instructor. 

The F/O playing catch— up during the simulators changed the whole 
emphasis of these periods from role-playing and habit pattern 
setting in routine ops to OJT in radio and instrument procedures. 

7 


I thought the initial training was fine, except for the 
simulator. I could have used more training on the Jep manuals 
and approach procedures. X was stuck as an S/O for six and a 
half years because of no hiring, and I had not flown all that 
long. 24 

We need more emphasis on crew coordination for < people like me 
coming from the single seat community in the military, especially 
with little S/0 time on the 727. 26 

The big item here is LOFT. LOFT helps a lot, but it takes a 
while for a new guy to figure out what to do and when. 28 

The training was excellent in all respects except one. a 

better, more sophisticated CPT would have been helpful. The 
ground school instructor and simulator instructor were superb. 

30 

I was quite satisfied with my DC-9 training. Simulators in 
general, though, need great improvement in the feel of the 
controls and the visual presentations. Both features, on every 
simulator I have flown, are over-sensitive when compared to the 
actual aircraft. A pilot has to develop two sets of flying 
skills, one for the real aircraft, another to satisfy the 
computers in the simulator. 39 

I enjoyed DC-9 training. I flew it a long while as an F/O, so it 
was like putting on a really great old pair of shoes. More 
emphasis may be put on less flying by the book in certain 
instances. New copilots will often try to be configured and on- 
speed at the marker, which really plays havoc with the people 
behind you, and makes the controllers job much harder. You can 
still fly smoothly and fast and get everything done by 1000 feet 
on a nice VFR day when you? re cleared for a visual approach. 

They <F/0s> don’t look out of the window nearly enough. (Nor do 
I, probably, but I try) . 40 

Excellent! All my ground school, simulator, and aircraft 
instructors were highly knowledgeable and helpful. I would like 
to see a more thorough review of Ops Specs material and perhaps 
some of the FARs. A periodic review of material in the AIM would 
be helpful. 41 
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I thought the DC-9 systems and simulator training was adequate 
(not good, but adequate) . Flying training (2 rides) was almost 
marginal • Every landing was a new adventure • X had only flown 
fighter-type aircraft prior to the DC-9. This put a lot of 
pressure on the line check captains that flew my IOEs. Most of 
my learning was accomplished on my IOE. 46 

For the most part the training was good. I don' t think the 
ground school portion can be any shorter. A lot of material i 
covered in a very short time, requiring a great deal of study 
time prior to classes in order to keep up. The cockpit trainer 
(DC-9) in ground school is helpful, but needs an overhaul badly. 
The flight training syllabus is very good. It does a good 30 b of 
preparing you for the evaluation. X felt that one of iy 
instructors could have been a bit more prepared when it came to 
briefing some of the pre-mission topics. 48 

More attention should be paid in ground school to flight 
instruments/autopilot/flight director. The instructors do a good 
i ob on mechanical parts , but tend not to understand the actual 
operation of the flying part or the actual uses of the automatic 
phases. 49 

The training was adequate. The major problem was in ground 
school: dated slides, systems comments by the instructor that 

were not covered in the manuals, the only reference now, after 20 
months since transition, is in my written notes. The slide- tape 
training environment (e.g. 767/757) is a better way to ensure 
that all info is properly documented and passed along to us. we 
shouldn't have to rely on the capability and quality of the 
individual instructor for the complete dissemination of the 
material. 74 


The training was great. I felt very confident after coming out 
of training. The training aids were poor or non-existent, we 
certainly need more and better CPTs. 80 
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4. Do you like the way the DC-9/MD-88 operates in the ATC 

environment? Please mention things you have trouble with, 
and things that work well, in dealing with ATC. 


MD-88 Crews 


TABLE V-3. Listed below are replies to Question 4 by two or more 
respondents (MD-88) • 

Map display (HSI) a real asset? aids situational awareness 
Too much time is spent programming in high workload periods; 

MD-88 is "too busy” 

Excellent at cruise; LNAV excellent? doesn't depend on VORs 
To much head-down time 

Difficult to descend? gets behind descent . 

ATC changes, crossing restrictions add too much workload 
WAV level-off at 10,000 not smooth 
Holding page (capability) helpful 
Automation is unpredictable 

Too many brief descents, interruptions from ATC 
DFW ATC operates poorly 

Controllers don't understand our "magic" 

Speed changes under autothrottle too abrupt 
Radar superimposed on HSI map excellent feature 
Problem of killing capture with V/S wheel 


The MD— 88 works well, but when things get busy it takes too much 

time to program the computer- It's just easier to hand fly some 

times- The autothrottles are the best part of the automation. 

They are extremely reliable, and free your attention to fly the 

approach more carefully. 9 

I like the way the MD-88 works with ATC, but I have trouble with: 

1) Shallow descent rate, 15 flaps, idle power, 180 kts. yields 
less than 1500 fpm <descent rate>. Sometimes this is not 
adequate at big airports. 

2) WAV is a nuisance below 12,000 since it decides when to 

slow to 240 kits. (10,000 foot transition). FMS OVRD is. 
good in theory, but the desired speed is not maintained in 
descent, the path is. t 

3) Updating continuously the heading bug m Map mode - 
busywork! 

Things I like: 

1) Nav display with weather projection capability. This is a 

great safety enhancement. . 

2) Nav display during vectoring for ILS approach. Situational 
awareness vastly increased. 
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3) Fuel calculations in FMS - a safety backup. 


10 


Cruise portion of flight is great - you don't have to constantly 
be following VORs, correcting for wind manually like m the DC-9. 
Descent profile helps in seeing if you will make <crossing> 
restrictions. Entering holding patterns with plenty of lead time 
is nice, however you still need to manually (mentally) figure 
these out. In a non- ATC environment, i.e. tower closed, call 
when you are on the ground, the CRTs are a must. They give you a 
great bird's eye view of where you are and where you want to go, 
and save you from landing at the wrong place. 11 

There are usually too many small descents, instead of 1-2 large 
descents. It usually requires too much work to allow the FMS to 
really be useful. Even if we do get the large descent, very 
often we will get airspeed changes in the descent, which are too 
much work to change in the FMS. 12 

ATC at DFW is very bad. They say keep speed up and then clear 
you for visuals from 11,000. MD-88 does not descend very well. 

ATL ATC keeps you informed about what to expect. DFW ATC is 
dangerous. Also have had them give crossing restrictions the A/C 
cannot make; then they get mad. 13 

Sometimes restrictions are given in a way that are not very 
efficient. This causes you to reprogram FMS (heads down mode in 
cockpit) • Also, not as much direct <clearance> as could be. 1 _ 
suspect that the controllers are not aware of the magic on board. 
For example, "fly heading 200 when receiving proceed direct." 

They are unaware that we can go FMS direct. Also, FMS system 
does not level and accelerate well at 10,000. It captures at 
10,000 feet, when levels off and then goes to climb power to get 
the 335 kts. 14 

Basically I like the operation. You must stay ahead of the 
magic, or it will lead you astray. You must be sure it is doing 
what you think it is doing. I have had several aircraft go from 
Nav Track to Hdg Hold for no apparent reason, and the only way to 
know is to constantly monitor the FMA. 18 

The MD-88 works well except for the descent phase. The FMS 
(VNAV) often gets behind in the descent. It is usually easier to 
adjust the descent using the autopilot. ATC changes speeds on us 
so often that the profiles are not workable and we lose the 
efficiency of the FMS programmed descent. The LNAV function is 
great. It allows ATC and the pilot great flexibility, especially 
when VORs are out of service. 

I particularly like the map display for maintaining situational 
awareness • I feel concern when the A/ C automation does . not 
respond flike engaging VNAV) only to find out that it did not 
take, yet I had anticipated that mode was engaged. Trying to 
translate ATC instructions directly into the computer and not 
getting expected responses (granted often operator error) , causes 
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a lot of head-down computer work. Altitude select knob set for 
100 foot increments a real poor arrangement. 21 

Generally speaking I like the way the MD-88 operates in the ATC 
environment. There is a problem on all automated aircraft when 
your departure includes a hold-down at 10,000. Once your 
autopilot goes into the altitude capture mode and begins to level 
off, and event which occurs at various altitudes depending on 
your rate of climb, the autothrottles start to retard to keep the 
speed at 250 kts. Once you level off at 10,000 the command 
airspeed bug jumps to 320-330 knots range and up come the 
throttles. If you were hand flying the airplane you would not 
have done this. Some pilots overcome this by going to FMS 
override and selecting a speed of 320—330 as the capture begins. 
The throttles then remain at climb power; however the airspeed 
does exceed the 250 kt. limit and the FAA has violated pilots for 
doing this. X think it is time for a regulatory change that 
would allow climbing aircraft that are cleared to 10,000 feet to 
utilize this procedure (or speed intervention on the 767 or 737) 
or anything that will allow the throttles to stay at climb power 
during this phase of flight. It would be a much smoother 
operation. Would it really be careless and reckless to operate 
at 300-350 kts. at some point below the magical 10,000 foot 
level? Another solution: don't clear us to 10,000 climbing. 44 

The -88 is a hand full. Digital programming takes a lot of time. 
Analog operation (DC-9) autopilot with basic throttles is a good 
deal easier to use in a fast -changing environment, and in fact is 
the method used on the MD-88 when things get busy. Below. 10,000 
feet one pilot is almost completely occupied working the radio 
with the other flying the airplane. 57 

I like the hold page. Gets you into holds easily. I also like 
the green descent arc in the map mode. No more mental gymnastics 
on crossing restrictions. I don't like a V/S wheel that controls 
both the V/S and the IAS, which when touched in the capturing of 
an altitude disarms the capture and selects a new pitch mode. 
Provides a lot of excitement. <See Question No. 1> 59 [L-1011 

F/0] 

FMS requires too much heads— down operation in terminal 
environment. Lack of tight integration in cockpit systems and 
automation increases workload. 62 

Things go well once you get the feel for the modes, and don't, try 
to set up the FMC below say 10,000 feet. ATC should avoid last 
minutes runway changes. New pilots get behind very easily in the 
automated cockpits. During parallel approaches ATC vectors you 
toward the localizer. They should always clear you to intercept 
it because often they get busy and run you across it, and legally 
you must just continue on their vector. 77 

I like the ability of the Nav display (ND) of the MD— 88 to show 
the relationship of the aircraft, nav aids, airports, and 
weather. This Is a major step forward is safety. When ATC gives 
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vectors in the terminal area for weather, the ND is invaluable. 
83 

The MD-88 is difficult to slow down, especially if required to 
descend at the same time. Autothrottle response is not smooth 
enouqh. If you are a little slow in a turn in the traffic 
pattern, or if ATC asks you to increase A/S by 20-30 kts., the 
autothrottles will jam on too much power — very uncomfortable 
for the passengers. 85 


DC-9 Crews 


TABLE V-4. Listed below are replies to Question 4 by two or more 
respondents (DC-9) . 


Need dual head radios 

DC-9 can do anything required 

Good radio practices are essential 

Low-tech works better than automation 

Precise navigation difficult? hard to track airways 

Poor trim adjustment in autopilot mode 

Controllers talk too fast . 

Need independent flight directors on each side 

Airspace is saturated 

Flight directors are antiquated 

ATC controllers are highly cooperative 

DC-9 very maneuverable, responsive to flaps/slats 

RNAV is needed 


4 

4 

3 

3 

3 

2 

2 

2 

2 

2 

2 

2 

2 


The DC9 works well except for being unable to get much point-to- 
point or direct clearances. If you are not RNAV equipped, you 
don't get the shortcut routes as often as RNAV aircraft. ATC 
provides headings depending on their workload. 3 

Troubles: holddowns, slowdowns, delays, saturated frequencies, 

less than capable controllers/ equipment, less than capable 
pilots, inadequate separation, untrained radio voices. Works 
well: flexibility, permitting deviations <route>, mostly sharp, 

clear, concise instructions or calls, forwarding weather 
information from previous flights, general courtesy. 4 

DC— 9 works well in ATC environment, but will not come down like a 
727 and won't go up like a 767. I have never before seen or used 
an flight director that doesn't have an altitude hold bar. I can 
fly the airplane better than the autopilot on a coupled approach. 
The autopilot has a hard time with gear/f lap/slat movement. X 
always thought the machine should be able to do better than me. 

I have to get everything stable so I can take over . All m all , 
it's a fairly versatile airplane. 6 

The DC-9 works well in the ATC environment specifically because 
it lacks the automation I loved so well in the 767. ATC is not 
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geared toward RNAV/VNAV capabilities, so these features can range 
from useless to handicap. The DC—9 is low— tech in a low— tech ATC 
environment. The most serious fault is the lack of positional 
awareness that could be greatly improved, not with a 767 map 
display, but with such simple additions as an HSI, and DME on all 
ILS frequencies. Having had DME to the runway m the 767 really 
emphasizes what a safety factor this is in the terminal area. 
There is occasional difficulty with ATC forgetting how busy a 
two— man crew can be. They always want us to understand that they 
were "on the land line", but often forget that we have an 
airplane to fly first, talk second. 7 

Yes, the DC-9 does a good job. It's a 1955 technology aircraft 
operating in a 1955 ATC environment. Please have controllers 
talk a little slower in the DFW and NYC areas. The faster they 
talk, the behinder they get. The system is getting overloaded. 

8 

The DC-9 is underpowered compared with other, newer jets. Often 
ATC will ask for unrealistic climbs. When we tell them that we 
cannot make it that high, that soon, we get vectored all over the 
place. Avionics are adequate for today's environment, although 
two independent flight directors would be helpful. Simplicity 
makes the airplane easy and fun to fly. No FMS to babysit. 29 

The DC-9 is OK. Our navigation ability is very limited as to its 
precision and overall capability ■ Numerous corrections are 
required to stay on course. We are rarely dead on an airway 
centerline. Much mental arithmetic and "how goes it" checks are 
needed to satisfy crossing restrictions. We cannot qo directly 
to a VOR much more than 200 miles away. VOR needle indications 
often waver back and forth making precise navigation difficult. 

39 

Yes, I like the way it operates, but if you have a problem, 
either in the aircraft or in the cabin in the terminal area - 
with bad weather and lots of traffic - you can really get behind 
trying to deal with everything at once. The DC-9 systems are 
simple enough that problems can easily be handled. Listening up 
before speaking; using your call sign always; don't get mad - 
people are probably trying as hard as they can to do right. If 
you have a delay or a go— around, so what? Come back and do it 
again. You can build situational awareness by listening to the 
radio and getting everything done before you get down into the 
terminal area. 40 

I am content with the way the DC—9 operates under normal 
conditions. However, during times of heavy workload, one 
additional factor such as an aircraft malfunction or passenger 
problem can tax the crew to the limit. An ACARS system would be 
useful in the DC-9. The flight director in the DC-9 is 
antiquated and "sloppy" compared to later generations. 41 

The DC-9 works well in the environments that the company sends us 
into. We do not go into LAX, DCA, or NYC. This relieves us of 
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some of the more unpopular cities to fly into. I also get the 
feeling that the ATC controllers are aware of our capabilities. 
They seem to issue clearances that are usually easy to 
accommodate. 46 

I think ATC must have been designed around controlling the DC-9. 
On other airplanes, they frequently set you up high. The unique 
problem of the DC-9 is tracking the airways. The controllers 
seem to be much more aware of the lack of precise adherence to 
airway centerlines. This of course is the result of our not 
having computer— generated displays m our cockpit. 6/ 

VOR/DME navigation is archaic. Point-to-point inertial through 
the FMS is more accurate and would (does) provide both ATC and 
the flight crews with better and simpler tools. The problem is 
with the mix of the two. If the ATC system were designed for 
RNAV, we could take full advantage of the current technology and 
simplify the entire system. How about requiring RNAV m the high 
route system? 7 4 

I like the fact that low automation keeps me involved with the 
flight, but automation would be nice to reduce the workload 
occasionally and reduce fatigue during long days. The most 
frustrating items in the DC-9 are: 1) flight director; 2) the 

PDI and RMI needles are all +/- 2 to 4 degrees, sometimes more. 
Some Centers are accustomed to and insist on all aircraft 
tracking a radial precisely . In the DC— 9 , there is no such 
precision. I think common sense, courtesy, and empathy go a long 
way in dealing with anyone. On thing that perhaps controllers 
could be aware of, particularly approach controllers , is that 
they sometimes ask us to maneuver in such a way that is 
uncomfortable to passengers. We tend to comply, partly due to 
vanity, and partly because we don*t want to be "punished with a 
delay due to failure to comply. 82 
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5- What can you say about overall workload of the DC— 9/MD— 88 
compared to other aircraft you have flown? Include mental 
workload, monitoring, etc. 


MD-88 Crews 

TABLE V— 5. Listed below are replies to Question 5 by two or more 
MD-88 respondents. 


Great amount of monitoring required, increases workload 
High workload; highest of any plane I've been on; 
higher than the DC-9 

No problem once you* re experienced on aircraft; 

initially difficult 
Mental workload higher 
Mental workload is less 

Checklist is too long, creates workload 

FMC must be watched 

757 is a superior airplane 

Workload the same as other aircraft; no problem 


There are more things to watch over than in a traditional plane. 
If you could trust everything to work as advertised, the only 
concern would be programming ( "typing" ) on the computer . 
Unfortunately when you least expect it, something goes wrong and 
you have to both fix the mistake or problem and also get the 
aircraft going where it should be. If you just had the normal 
scan and flew the airplane it would be simpler which of course 
you can always revert to if you want. 9 

Checklist is too long. Initial setup at the gate is relatively 
long (i.e. entering fuel into ACARS, enter ZFW twice, entering CG 
and ZFW CG) . However, many other chores are eliminated (EPR 
calculations, gross weight). In flight the workload is lightened 
except for the checklist length. The revision of our checklists 
(12-15-89) is a big improvement. Mental workload is greatly 
reduced thanks to the FMS . However, the FMS does occasionally 
miscalculate and it needs to be watched fairly closely. 10 

The highest workload of any (transport) aircraft I have flown. 

The workload on the ground is heavier than any aircraft I have 
flown. Monitoring can be a pain. There are too many 
combinations of switches that can get you in trouble. 11 

From T/O to landing the workload on the -88 is less. If feel 
that in the terminal area, the use of the FMS with the autopilot 
is too much work. It is far easier to manually control the 
autopilot or hand fly. 12 
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You can’t take your eye off for a second* X enjoy the workload. 
Otherwise I would get bored. 13 

Harder to stay in the loop - no need to get chart out and check 
radials. This can cause boredom, causing pilot to be out of the 
loop. It is harder to be a passive monitor than an active one. 

I try to check radials and use charts to prevent this. 14 

In the MD-88 you can do much more, but automation increases your 
workload due to the increase in monitoring duties and FMS 
programming. 15 

Under normal circumstances the MD— 88 workload is equal <to the 
DC— 9> . But, at time of high stress, if you try to stay up with 
what is going on and keep using the magic, the workload is much 
hiqher. In other words, the magic relieves workload only if it 
is set up ahead of time. I am not sure it really reduces mental 
workload because you cannot sit and allow it to function without 
close monitoring. 18 

The MD-88 has a high workload. The FMS is a great tool but 
demands a great deal of attention. You have to constantly be 
aware of the mode it is in, and it often will switch modes on 
you, LNAV to Heading Hold, for example. With the influx of new 
copilots it is an added task to teach the FMS and how you want it 
used. There seem to be different attitudes (both m training and 
among other captains) on how to use the system. The automated 
MD— 88 definitely increased my workload compared to the DC-9. 19 

Compared to the 767/757, I feel that the MD-88 has to be watched 
carefully. I feel that the two-man crew (any two-man crew) 
creates anxiety in that we’re often both busy and not atl© 
monitor each other. On the three-man crew, I felt that there was 
better cross checking of the other pilot. 21 


The workload on the MD-88 is a factor of 2 to 3 times great than 
the DC-9 on the ground (preflight, taxi-out, checklists) . Once 
airborne, workload may be slightly less if little programming is 
required, or maybe more if changes come up which require 
programming. 34 

Workload is no problem. I have to admit, when I was flying ona 
three— man crew during the initial discussion about . glass cockpits 
and two— man crews I was skeptical. I became a believer about 
half way to Los Angeles on first leg of my IOE as a 767 F/0. I 
have since had to fly technologically disadvantaged three-man 
crew aircraft, and I did not enjoy it at all. 44 

During smooth, normal operations the workload is less<in the MD— 
88> since less "thought" (e.g. tracking radials, working out 
drift corrections, etc.) is required. During emergencies/ system 
problems a two-man aircraft involves more work and crew members 
are unable to effectively monitor each other since one person 
must fly and the other work the problem. In the MD-88, 
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preoccupation with programming the FMS during these emergencies 
can lead to task saturation, even though FMS programming is only 
slightly helpful. 55 

Overall the workload seems to be greatly reduced. In the 
departure and terminal stages, this is essentially true when the 
full capabilities of the FMS are understood and utilized. During 
the first several months on the MD— 88 , though, the FMS can be 
overwhelming and greatly increase workload. 

Flvina the DC-9 was a lot simpler, but required a lot of head 
work. You were more involved with, the flying. Your brain was 
the computer, for better or for worse. In the MD-88 you do a lot 
more monitoring, and do it rather poorly . . 1 "°uld always want to 
be looking at an approach plate when we didn t capture an 
altitude. 59 

The autopilot/ flight guidance system is poorly integrated on the 
MD-88. The older B-737 SP-177 autopilot/ flight director is much 
easier to use and much more reliable. The workload in the 
terminal environment and during preflight is high <MD-88>. The 
overall cockpit layout is poor - the hazards of derivative 
aircraft. 62 

Initially the workload is quite difficult. After about 50-100 
hours workload becomes better and at the 200—300 hour point 
workload will decrease. Monitoring is very important and you 
must train yourself to continue to monitor, especially when you 
think all is well. Remember, in God we trust, everything ^ else we 
check. In an automated airplane, I recommend check twice. 77 


DC-9 Crews 


TABLE V-6. Listed below are replies to Question 5 by two of more 
DC -9 respondents. 

Workload moderate, manageable, same as other aircraft, 
no problem as long as conditions normal 
Workload is higher, or excessively high in DC-9 
Workload is excessive below 10,000 feet 
Third man is needed during abnormals? workload 
higher in any two-man plane 
Workload is fatiguing 
No "magic" - less to worry about 
First officer is overworked 
Monitoring load is high 
Mental workload is high 


DC— 9 workload is as much as any aircraft in our fleet. With new 
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copilots it increases the captain's monitoring workload. New 
copilots should have more company experience before checking out 
as initial F/O. 1 


When everything is going well, the workload is fine. It's just 
when outside factors such as weather, maintenance problems, or 
passenger problems enter the picture that it can create stress. 
The DC- 9 is a hands-on operation, so you don t have the computer 
to worry about. I find that when the workload gets hectic, 
actually hand flying the aircraft helps me deal with the 
situation better. 2 


Man is better at working than monitoring. To me, the new 
generation of aircraft should advise the pilot of possible errors 
in a passive manner, and possibly provide guidelines for action. 
Computers are better at monitoring, and should be utilized to 
assist the pilot in his decision making. Monitoring can be very 
boring work and even a distraction. Remember, "quiet and dark 
are ideal for sleeping, too. As to below Cat II approaches , why 
deliver people to a destination where they can't see to get home. 
4 

Much of the DC-9 workload in arrival/departure area is a very 
high mental workload. In the 1950 cockpit and the 1990 
environment, we must do most of our decision making . without . 
information that the glass cockpits have. No wind information, 
no visual picture of the airport and our position. CRTs show far 
more information than we have in the DC— 9 cockpit. Autopilot, 
autothrottles, and altitude capture are sure a help m arrival 
and departures • 5 

On every airplane I have flown, one crew member does more work 
than all the others. The F/O on the DC-9 does all the engineer s 
work, all the F/O's work, and half the captain's. I try to help 
as much as I can, but sometimes get loaded down and I can't watch 
him <F/0> as much as I'd like to. I like to fly the airplane to 
cruise altitude, then hook up the autopilot, and later fly the 
plane from 10,000 to landing. I think this lessens my workload. 

6 


Stick and rudder and navigation workload is higher , but button 
pushing workload is lower, than automated aircraft. Overall 
workload on the DC-9 is lower than the 757, but the tasks that 
remain are not as much fun because they include the menial tasks , 
such as chart study, course setting, etc. It is too simplistic 
to stack up tasks on each side of a scale. Higher workload on a 
757 can be more relaxing and enjoyable if the tasks themselves 
are less tedious and repetitive. Comprehensive warning systems 
greatly reduce the mental workload because a pilot does not feel 
the need to constantly recheck routine items. Energy can be 
directed toward flying the aircraft. 7 

The workload of the DC-9 is similar to the 727 and DC-8, very 
manageable until an abnormal situation occurs. Then the 2 -man 
versus 3— man crew differences become obvious. With a 3— man crew 
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there is always someone who can devote full time to flying the 
aircraft. In a 2-man crew, in an abnormal situation (mechanical , 
passenger, etc.) generally 1 1/2 men are dealing with the problem 
and 1/2 man is flying the aircraft. 8 

<The DC-9 has> fewer magic components to babysit. This makes 
things simple. It forces you to stay involved with the airplane. 
If too many things were automated, you would become detached and 
miss things. Overall the workload is moderate, depending on the 
terminal environment. I enjoyed "flying the airplane with my 
mind" and figuring out climbs/ descents etc. 29 

The workload on the DC-9 is greater than it was on the L-1011. 
This is partially caused by the fact that the DC-9 is two man 
crew (vs. 3 for the L-1011) and normally flies more legs per duty 
period. But the DC-9 also has a higher mental workload than the 
L-1011 due to the lack of automation on the DC-9 . The DC-9 
requires constant attention to navigation. There are no 
computers to automatically fly a programmed route. Much mental 
arithmetic is needed in the approach phase to compute start-down 
points, and to satisfy altitude crossing restrictions. On a 10 
to 12 hour day involving six-plus legs, it definitely affects the 
fatigue factor. 39 


DC-9 - higher workload due to shorter hops. More stops equals 
more dealings with ATC, passengers, fuelers, etc. More time down 
in light plane country. I get pretty tired after a bad day 
(weather, all instrument approaches to minimums, etc.). The 
four— day trips we have are notorious. You really have to watch 
it on the fourth day. You can easily make errors. 40 


I don't find the workload a burden. Naturally a new copilot has 
his hands full and it takes a while for him to feel comfortable. 
The workload is higher in any two-man plane than the 3 -man. witn 
the DC-9 being a manual aircraft, once you have everything done, 
you just fly it. The automated aircraft takes more work 
reprogramming. Once you are flying, unless you get ATC changes, 
the automatic aircraft will do everything for you. My friends on 
the MD-88 say that because the computer says it|s right, many 
people don't cross check the automation to see if the information 
is correct (for example, are they really going to make a speed 
and crossing restriction) . 63 


There's no doubt that the DC-9 is tough duty compared to other 
airplanes. Lots of legs, lots of small, poorly equipped 
airports, lots of new employees, limited avionics and airplane- 
related equipment. The good part is that it's really flying. 

Lots of challenges with proven, albeit old, equipment, and eager, 
young co— workers. What could be better? 67 

I believe for the workload for the captain is a lot less than 
other aircraft, but for the F/O it is a lot more. 80 


102 



6. List the features or modes of your aircraft flight guidance, 
instrumentation, and avionics that you like and dislike. 


The results of this question are given in Table V-7 below. 
Following the tabulated responses are verbatim responses where 
these were included by the volunteers. 


TABLE V-7 . Likes and dislikes in cockpit equipment. 

LIKES: MD-88 Crews 

Item 


Times 

Mentioned 


Map mode of HSI 

HSI map and weather radar on one display 
Airports and nav aids on HSI map 
Flight path on HSI map 
Map plus DME 

Flight Management System (FMS) in general [1] 
Information available from FMS 
Hold page , . 

Autoland Cat II and III capability 
Autothrottle (esp. T/0 and cruise) 

LNAV (NAV) 

VNAV 

FMS override button 
Auto go- around 
Direct intercept page 

VOR designator (3 letters) to establish waypoint 
Various levels of automation available 
NAV-autopilot-autothrottle relationship 
Vertical speed mode to climb in turbulence 
Reliable autopilot 


13 

9 

3 
1 
1 

10 

4 
6 

5 
4 

3 

4 
2 
1 
1 
1 
1 
1 
1 
1 


Glass cockpit in general; EFIS displays 
FMA panel . . 

EPR calculation regardless of anti-ice use 


Color radar 
ACARS 

Fast/ slow bug on PFD 
ILS display 
ADF display 
ETA to fixes 


Dual head radios 
Engine power 
Automatic strobe light 
Autobrakes 

Transponder auto shutoff after takeoff 
Pressurization 
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DISLIKES: MD-88 CREWS 


Item 


AHARS (unreliable? not up to IRU) 

VNAV descent (rough? unable to maintain airspeed) 
VNAV level off at 10,000 feet (power surges) 

ND obscured by yoke 

Altitude select knob (100 foot adjustments) 
Mysterious loss of alt hold and alt capture 
Altitude control in level-off 
Altitude alert window (numbers change) 

Map failures due to navaid loss 
FMS hard to get used to 
DFGS in general 

FMS too complex (e*g. hold page) 

Autopilot-FMS interlink cumbersome 
FMA Info should be on ADI (like 757) 

Can't auto tune VORs 


Autothrottle response 

A/T slow to respond to level off 

Speed modes awkward 

Autopilot 

A/T on landing 

Assigned airspeed doesn't go into profile calculation 
Difficult to transition to IAS mode smoothly 
Vertical speed wheel 


Too many FMC (and map) failures 
Computer slow to respond 

Burned out lights in engine display, FGS panel 
Lack of logic in switch patterns and movements 
MCP design 
ADI screen in fuzzy 


Air conditioning system 

Flat plate digital instruments for engine parameters 
MCP design 

DFGS should have self -illuminating buttons 

Time spent programming (head in cockpit) 

Updating heading bug (company policy) 

Too many ways to do the same thing 

Too many buttons to press to do some things 

Too many repeated inputs (e.g. ZFW) 

VNAV below 12,000 


Times 

Mentioned 


7 
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LIKES : DC-9 CREWS 


Times 

Mentioned 


Item 


Simple operation? basic airplane; easy to understand 

H Everything" - good generic airplane 

Basic airplane - allows you to keep up skill level 

Avionics in general 

Reliability 

Flight director (" simple") 

Autopilot ("simple” , "predictable") 

PDI reciprocal heading feature 
SCAT 

ILS - A/P mode 
Heading select 
Approach coupler 
Engine instruments 
VOR/DME 

Manual throttles 

"steam gauge" <traditional dial> instruments 
Heading bug on flight director 
ILS ("straight-forward") 

Hand flying 

Smooth roll control 

Landing characteristics 


8 

3 

1 

3 

2 

8 

6 

6 

3 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


Comments 

The DC-9 works fine if you understand its limits. 37 

All facets of the avionics, instrumentation, and guidance are 
SIMPLE! The autopilot doesn’t fly the pilot: the pilot has to 

fly the autopilot. VOR/DME information seems to be enough to get 
around in our domestic U.S. system. I'm not interested in INS or 
ground track information. I really like flying the DC-9. 

The DC-9 is a good basic aircraft that will do anything but 
autoland. Because it is a basic airplane, it keeps your flying 
skills up. 63 

The good thing about the DC-9 is its simplicity. Once you've 
made the trip back down memory land and gotten comfortable again, 
it's really a remarkably reliable, simple airplane. And best of 
all, it flies great. 67 


105 



DISLIKES : DC-9 CREWS 

<Note: most of the "dislikes" listed by DC-9 crews were items or 

features that are not present in their aircraft. These are 
listed in the first segment of the table below>. 


Times 

Item Mentioned 


Lacking , or would like to have: 

HSI (in place of RMI and PDI) il 
Autoland “ 
Pitch reference on F/D j- 
Altitude hold bar on F/D i 
VOR annunciation in F/D mode 1 
Light test feature in F/D mode 1 
Altitude capture on A/P 5 * 
Pitch command on F/D in level flight 1 
Wind or TAS readout j- 
Go-around mode on A/P 1 
RNAV ("for fuel savings") i 
Angle of attack indexer ("for windshear escape") 1 
Third VHF com to monitor company radio 1 
Dual DME displays for each pilot 1 


ADF (non-digital) . 4 
Weather radar (need color? antiquated? erratic) 3 
Flight director 3 
Cockpit antiquated in general 3 
Autopilot ("sloppy", unresponsive in some modes) 2 
Autothrottles 1 
ADI (too small) J 
VOR mode on A/P 1 
Engine instruments 1 
Gear horn cutout only on F/0*s side 1 
Hard to equilibrate VHF volume esp. over 10,000 1 
Can't hear bell when F/A calls 1 
Would like redundant and independent A/P and F/Ds l 
Pitch control too sensitive on descent 1 
Sometimes skids when A/P engaged in level flight 1 
Instrument lighting 1 
Can't engage A/P in a turn 1 
ATC demands more than a DC-9 can do 1 
ILS needles are in poor position 1 
Too many lights are illuminated on the ground - hard 

to catch important one 1 
Altitude alert 1 
Hard to remember which compass inputs to what 1 
Lacks the basic "T" arrangement due to PDI 1 
F/D and A/P don't always agree on coupled approach 1 
Approach coupler is not smooth 1 
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Pressurization is a problem is not set correctly 


Can't see VHF No, 1 freq. when captain uses A/P turn knob 1 
No. 1 DME and altimeter hidden by yoke 1 
Slat extend light hidden from view [1] 1 


[1] Doesn't say by what. 


Comments 


1950 's cockpit in a 1990 environment. 

The whole cockpit of the DC-9 with respect to flight guidance, 
instrumentation etc. is marginal. The reliability of our DC— 9s 
is superb. While I may not like some of the instrumentation, at 
least they always work. 
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7. 


If you have flown both a traditional and a glass cockpit 
aircraft (MD-88 or 757/767), please give us your insights 
into the differences between operations in the traditional 
and the glass cockpit, (If not, please leave blank) . 


MD-88 Crews 

I have flown traditional aircraft (KC-135, T-37) in the Air Force 
and now the MD-88. The biggest advantage to the glass cockpit is 
the amount of information available to the pilot from the FMS, 
and the autothrottle and autoland capabilities. The disadvantage 
is the time spent inside the cockpit trying to program the FMS, 
when often times it would be easier to fly the whole plane by 
hand without the neat gadgets. My biggest concern with the glass 
cockpit is the erosion of basic flying skills in instrument 
conditions. We rely so much on autopilot (because we are 
encouraged to give a smooth ride to the passengers) that the 
basic crosscheck of flying instruments, and eye-hand coordination 
skills, become rusty along with the confidence to fly an 
instrument approach down to minimums. 9 

I rely on the FMS map without cross-checking the charts. Every 
now and then the map will be off by up to 1/2 mile, as verified 
by the outer marker. Situational awareness is much better in a 
glass cockpit. 10 

I like the glass cockpit. The CRT is tremendous in helping us 
know exactly where we are and how much we really need to deviate 
in weather. The bird's eye view it gives us is fantastic. The 
flight guidance system and FMS still increase rather than 
decrease workloads at the times you need their help the most. I 
liked flying the glass cockpit because it was a challenge, but it 
definitely is not a workload reducer in its present format. You 
also have to work at keeping up your basic flying skills or you 
lose them. I still back up everything with VORs and radial 
selections. If in doubt, I still believe the VORs. 11 (L— 1011 

F/O) 

The glass cockpit aids the pilot in finding a field he is 
unfamiliar with at night. I would never go back to traditional 
cockpits if I can help it. 13 

I flew both the DC-9 and the MD-88. You can do much more with 
the FMS and the DFGS and autothrottles . But there are moments 
when you workload is actually increased on the MD-88. Also, 

FMS/ auto throttle responses are sometimes very slow or abrupt. 

They need to be updated to smooth, reasonable response times. 15 

The glass cockpit has significantly more information available to 
the pilot on few instruments. The glass cockpit or advance 
automation is hard to use or program when unusual situations come 
up, such as holding at a different fix, or routing changes once 
the computer has been programmed. 16 (DC-9 Captain) 
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The glass cockpit has some very, very nice features , such as the 
nav display, which makes route planning, weather avoidance, and 
ever to some extent route changes easier to visualize and 
complete. I particularly like the nav display during an ILS 
approach when the whole picture is being displayed and your exact 
position is easily determined. X do not like the fact that the 
same nav display with go D/R and then "nav fail" at the worst 
moment. Then the workload required to recover the N/D is usually 
too high and the rose or arc displays must be selected. In the 
terminal area, the workload can be very high if you try to keep 
up with the FMS. Usually it is better to set it up for the final 
approach and use the "straight" autopilot until then. I also do 
not like the fact that at least half the nav display is hidden by 
the yoke. X cannot emphasize enough the absolute necessity as 
captain of being certain that the FMS is doing what you think it 
is doing. 18 


In the traditional aircraft, all attention in the critical areas 
of flight are on the primary flight instruments, and if it s yFR, 
the outside environment. To use the automated cockpit, a certain 
amount of the attention is diverted. It has to be, in order to 
run the automated cockpit. On takeoff the PNF has to look in the 
cockpit to push climb power, VNAV, and to run the flaps up at 
1000 feet. In the traditional cockpit, the PNF can do his duties 
and still look outside. In the approach environment the PNF has 
to have his head in the cockpit much more in the automated 
cockpit. I feel that the automated cockpit is not designed for 
VFR. Your outside awareness is much better in the traditional 
cockpit. 19 


I believe I maintained a better situational awareness in the 
traditional cockpit. I find that I frequently expect the magic 
to take care of me and I allow myself to be distracted from 
flying the airplane. 21 

Traditional is much easier and simpler to operate. .. few system 
require "programming." Set VOR course and frequency — what else 
is there? It requires you to stay in the loop and do more head 
work (updating frequency and course, planning descents, 
visualizing a holding pattern or best headings to go around a 
storm. On the negative side, if you get out of the loop, there 
is no pre-programmed course or descent to back you up. The basic 
difference is that a glass cockpit provides a tremendous amount 
of information, some of it useful, some of it distracting. 
However, in order to get the useful information, the system has 
to be programmed en route. This takes time (workload) when time 
may be of the essence. The interface between crew members and 
the systems is a major factor in how the system will be used. 

The interface on the MD-88 has not been designed as well as it 
could be (767 guys can't wait to get back to that aircraft) . 34 

From an operational point of view, there are few if any real 
differences with the exception of the map <HSI map mode>. While 
this is very useful, it promotes a loss of ability to "visualize" 
the aircraft position. With respect to the FMS, I strongly 
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believe that a solid foundation is needed in terms of basic 
flying skills- This is difficult, if not impossible, to get by 
initially flying an aircraft like the MD-88. 35 

1) Glass cockpit is always 2— man. In weather with an emergency 
you approach (sometimes exceed) task saturation- 2) High level 
of automation can leave pilot out of the loop since humans are 
not good at just monitoring (without coupling monitoring to 
action), and you get bored and distracted- That 8 s fine as long 
as the system is working properly, and it's been programmed 
properly. But if these two prerequisites are not met, it 11 bite 
you. 3) Glass cockpit - a pre-condition exists (I know cause 
I*ve done it) with analyzing why the aircraft isn't doing what 
you thought you told it to do. So you try to reprogram it. All 
the while the error is uncorrected; e.g. it does not intercept 
the ILS , so you check that it is armed, that the correct freq is 
in, etc. Meanwhile you are shooting through the localizer or 
getting high on the glide slope. 4) Traditional cockpit - much 
less flexibility than glass cockpit, e.g. lacks RNAV, more 
difficult to remain position oriented, requires more attention . 

55 

The workload of the PNF is greatly reduced in the glass cockpit 
due to the fact that he doesn't have to look up freqs/ courses, 
etc. The ability to know your exact position is definitely 
comforting. This is useful during visual approaches into 
unfamiliar fields. 56 

Glass cockpit are definitely higher workload environments. I 
like them for a perverse reason - I like gadgets, as I suppose 
most pilots do. Crew coordination gets to be a problem on an 
aircraft with a DFGC because of the many different ways to 
operate that accomplish essentially the same thing. This 
presents more areas of disagreement between crew members as to 
which technique is better. X frankly think that the traditional 
cockpit is a far safer operation, but the glass cockpit is the 
future because CRTs are cheaper to maintain than . conventional 
instruments, and this is a "bottom line" driven industry. 57 

In the glass cockpit you rely more on the computer, and the 
company pushes you to do it that way. They say you should back 
everything up manually, but if you did you couldn't handle the 
workload. I felt that my flying skills were better than flying 
the traditional planes, but now I make an effort to turn off the 
magic every once in a while to remain proficient. 58 [767 F/O] 


DC-9 Crews 

<Note : Most DC-9 crews had not flown glass cockpit aircraft, and 

hence could not answer this question. Only ten replies were 
given. Nine of these are reproduced below; the tenth was 
incomplete. Of the ten replies, eight had previously served as 
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first officer on the 767, one was now flying as first officer of 
a 767, and one captain had just completed 767 school. None of 
the DC-9 pilots in the study had MD-88 experience (see Figure II- 
2 and II— 3) . The past or present 767 experience is noted in 
brackets at the end of the comment •> 


The traditional A/C can be set up for departure and changed much 
easier and quicker than the glass A/C. Once we ® n <L2 n 

our way, the glass cockpit provides much more information and 
closer tracking tolerances than traditional A/C. It s harder to 
change route in the glass cockpit. I prefer the large HSI of the 
767/757 over the smaller one on the MD— 88. Approaches are easier 
with the extended runway centerline and less chance for wrong 
airport landing under visual approaches with the glass cockpit. 

3 [former 767 F/O] 


You stay more on top of situations and airspace around you in the 
traditional cockpit. Glass cockpits are fine if all goes as 
planned, and the programmed information is correct. It is 
disconcerting to have your head down reprogramming when you 1 re 
descending into busy airspace. Likewise, to delay departure 
because you are re- or re-reprogramming your magic for ATC 
changes, or disagreement with its internal workings. Glass 
cockpits can be very boring - to distraction - when you re 
cleared direct in the dead of night. But they can ease the 
workload when you know what lies ahead. I prefer doing instead 
of watching, yet I do appreciate the split-second support the 
magic can provide. 4 [former 767 F/O] 


The glass cockpit gives one so much more infoipnation in one 
picture that mental time can be used on the big picture. a 

conventional cockpit much of the time is spent visualizing where 
you are and where you will be. The glass cockpit gives you the 
entire picture, as well as wind vectors, climb and descent 
crossings. Automation in glass cockpit ^ allows you to manage the 
aircraft and spend time planning. Traditional is more hands-on, 
so while you are trying to stay ahead, you must spend quite a lot 
of time monitoring what is happening now. 5 [former 767 F/O] 

The glass cockpit has a much lower workload en route . The tedium 
of frequency changing, course setting, estimate making, etc. is 
all gone. The downside is that the pilot can drop out of the 
loop mentally. This is not so much a problem at cruise, but 
finding yourself out of the loop on an approach when computer 
dumps can be disastrous. The temptation to re-program at this 
point can be overwhelming and also very dangerous. (We don t 
emphasize enough the choice of switching to manual nav at this 
point.) In the traditional cockpit, there is less automation, 
and thus less risk of finding yourself lost. One must stay in 
the loop just to get the basic job done. . .but overall capability 
is decreased at a price. On balance, the glass cockpit is highly 
desirable and a great safety advance, if training emphasizes the 
dangers' of over— dependence on the displays • 7 [Former 767 F/O] 
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I just checked out on the 757 /7 67 in February <questionnaire 
received March>. I have to qualify my answer with the fact that 
have one month line experience on the 767 and things are 
f£ 11 7 m 2 ew me. The FMC programming is not as compatible with 
the ATC system as it should be. Too many times ATC clearances 
are not as exact as FMC programming. Rather than giving a direct 
clearance, ATC gives you an airway intercept. It takes more 
steps to program - it's easier to pick up a map. The FMC is 
great for en route clearances and low density areas. There are 
m?° changes to make in high density arrivals and departures* 

The VNAV is not very smooth in making crossing restrictions. 24 
[New 767 F/O] <Note : prior to the completion of this study, the 

company recognized this problem and put more emphasis on choice 
of automation level in the training program>. 


I have just finished training on the B-767, but have not yet 
started flying it on the line. I do have some observations 
though. I like very much the technology of the 767. However I 
think an error has been made in the philosophy of flight 
training's approach to the use of that technology. We are 
basically instructed to use the auto systems all of the time. 

Here I disagree. While we need to be thoroughly familiar with 
the use of all the systems on the glass cockpit, we should also 
be encouraged to often fly the aircraft using only raw data (DC-9 
type) instrumentation. I think with all the emphasis on 
automation there is a subconscious tendency for all pilots to 
feel somewhat removed from the operation of the aircraft as the 
computer system” takes over. Complacency could be a result. 

For sure greatly degraded basic instrument flying skills are a 
result. 39 [767 captain just out of training] 


The single largest drawback to the glass cockpit is the amount of 
time required to feel comfortable and proficient with the 
systems. Once you learn to scan and assimilate all the wealth of 
information supplied by the glass cockpit, it becomes highly 
beneficial. The autoland capability of the 767/757 is truly an 
asset (likewise the L— 1011 autoland) • X miss not having such a 
system on the DC-9. 41 [Former 767 F/0] 


For *. t ^ e J _ f i rst , 2 ? Y ears °f my flying career my experience was 

to trad i t i onal cockpits. I felt comfortable 
knowing that whatever needed to be done would only be done if I 

eas Y to understand the concepts and equipment and 
the major challenge was to develop your own physical skills to a 
ere they measured up to your own lofty expectations. Now 

?i?rr£+- Capt ?i n ' P rimari ly focused on the safe completion 

or a flight, rather than impressing myself or someone else with 
finesse and perfection in physical flying skills. 

1 like the assistance that the automation 
gives in that it allows more attention to be given to the ”bicr 
picture”, rather than concentrating on physical details. 

««.,^^ The 4 - pro ?i em obviously is that on rare occasions, there is an 
equipment malfunction, off— route clearance in a busy ATC 
environment, or weather problem that necessitates reversion to a 
manual mode. This gives you a great deal of simpatico for a fish 
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suddenly hauled out of water* Charts fly, blood pressure rises, 
and confusion reigns supreme. With 20 years of experience flying 
manually, I was able to cope with the occasional misfire. My 
concern is with people who in the future will have less and less 
of a reservoir of experience to fall back on. 

My other concern is with the transition period into the 
glass cockpit. Six months and 300 hours into the transition 
period, I felt extremely comfortable with and confident in the 
new technology. In fact, I would still go back to the 767 m a 
minute if seniority allowed it. The first six months was a 
period of slow assimilating the new way of flying. I felt like a 
world class marathoner trying to ride around on a bike and 
getting tired. Different skills were required to do the same 
task, and it seemed quite frustrating and confusing. 

Another concern is that making the airplane do what you want 
it to do by pushing buttons becomes almost an end in itself . You 
have some difficulty ascertaining that line where you need to 
give up on controlling the magic and concentrate . on controlling 
the airplane by whatever means necessary. This is also a concern 
below 10,000 feet where you can easily become enthralled with 
pushing buttons and monitoring the magic and forget to 
concentrate on such real hazards as uncontrolled aircraft and 

flocks of birds. _ ^ _ .. 

I only flew the 767 for one year, but toward the end of that 
time, after 600-700 hours, another potential problem became 
apparent. After you spend six months figuring out how to 
efficiently operate this marvelous machine, you start developing 
a deep appreciation and respect ... almost awe... of it. You 
monitor it hour after hour and it functions perfectly and you 
soon realize that it can fly longer and more perfectly than you 
will ever be able to do. After a while it's difficult, bordering 
on the impossible, to monitor it the way you would a fellow crew 
member. After all, you know that a fellow crew member is going 
to foul something up once in a while, so you keep alert knowing 
that you may need it for ammu nition for the times when he catches 
you in a foul-up. It makes a nice little mental game and keeps 
you both sharp. With the computer, there's so few glitches that 
the mind finally just gets overwhelmed with monitoring and goes 
into the ignore mode in spite of all your best efforts to the 
contrary. 67 [former 767 F/0] 

By far the biggest advantage of the glass cockpit lies in the HSI 
map mode. The ability to orient ones self at a glance instead of 
relying on the mental gymnastics of bearing and DME, which 
requires constant updating (and the added burden if the VOR is 
co- located with the airport) frees one to concentrate on other 
aspects in the approach environment. I think the company should 
emphasize (encourage) the importance of each pilot hand flying a 
raw data approach at least once during every rotation — especially 
on the 767/757. In my three years on the 767, I was amazed at 
how few pilots went back to the old compass rose and flew raw 
data. It was a h umb ling experience every time I did it, but it 
was a good reminder that the "old ways" are still there as a 
backup. I had one experience with a bad DME signal in Boston 
that skewed our map by over 15 miles on an approach to R/W 4R in 
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low weather conditions. Otherwise, the system worked flawlessly. 
74 [Former 767 F/O] 
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8 . 


Please feel free to add anything else that you think we 
should know about the human factors of your flying 30b, 


MD-88 Crews 

Overall, the MD-88 is a big improvement over older generation 
concepts. If the pilots are not prone to complacency, it is much 
safer in the MD-88, as well as more efficient. It is up to the 
pilots, however, not to let their attention lapse to a point 
where they let the A/C do things that are not intended. It is 
easier to stay ahead of the MD— 88 than on other A/C, but it is 
not automatic. 10 

Stress keeps going up. I am afraid of making any kind of mistake 
for fear of losing my livelihood. I am human and we humans are 
not error free, yet we keep going with the opposite assumption, 
and when we do make an error, we are therefore punished even 
though there may be a reason for us doing this. We look at what 
error was done instead of trying to understand why it was that we 
were placed in a situation where the likelihood of an error being 
committed was greater. Radios are the weakest link in our 
system, yet we run our operation as if they will always work. 

When you really need the readback procedure to work on ATC, which 
is when you are busy, it's a joke. When you are busy the 
controller will usually miss a wrong readback. He too is task- 
saturated and is no longer listening — he is thinking of the 
next command he must issue. 11 

I would like to see a study of the duty day, including: 1) the 

number of legs per duty period; 2) rest breaks during a duty 
period. Some of our trips have 12 -hour days with at most a one- 
hour break between legs with an A/C change. This leaves no time 
to eat during a long duty period, especially in bad weather? 3 ) 
rest breaks between duty periods. 18 

I feel that there is a definite difference in the copilots that 
have flown copilot on traditional cockpits and then moved to the 
automated? as opposed to copilots that have just flown the 
automated cockpit. The former are more aware of the outside 
environment and flying the airplane. The latter are much more 
concerned with using the automation efficiently, but forget that 
we don't fly by ourselves. I also feel that we need to 
constantly practice flying airplanes. We need to make raw data 
approaches and takeoffs. That is how we get our feel for the 
airplane. I hope that the new automation does not become a 
crutch rather than a tool> 19 

I strongly believe that the 3 -man crew was lost to purely 
economic considerations, at a real safety loss. We gave up the 
redundancy of human factors. When the "going gets tough for 2 - 
man crews, the pilots often operate almost independently, unable 
to monitor each other. On three-man crews, I always felt that 
one pilot could concentrate on "aviating" and the other two 
pilots could run the appropriate checklists, trouble-shoot, etc. , 


115 


while double-checking on one another! The MBAs have won again! 
21 


With the advent of more complicated electronics and nav aids, we 
are advocating that the pilot not flying make all changes on the 
flight guidance control panel, flap position, radio frequencies, 
lights, ignition, anti-ice, etc. This further complicates 
matters for the pilot flying (hand flying) in that it takes time 
(albeit a short time) for the brain to put a name on a particular 
control to be moved, verbalize the control and movement. Then 
reception, processing, and reaction time of the pilot not flying 
come into play . The comment from above is think ahead . If we 
could always foresee the future, instantaneous reaction of the 
pilots would not be necessary. 22 

Pilots need to remain proficient in all levels of automation with 
respect to the aircraft. A pilot needs the ability to go to 
“Plan B'* in the event of any kind of automation . failure . _ Company 
policy should promote this philosophy of periodically using no 
autothrottles, no flight director. This would help keep those 
skills in tune when the automation fails. 35 

The absolutely worst human factors problem in our business today 
is the continued scheduling of crews with late night and early 
morning duty periods within the same rotation, as well as . the 
month* s schedule. For example, I had one trip when I arrived in 
California at 1:00 a.m. (4:00 a.m. EST) . Then a day later I had 
to get up in CVG at 4:15 a.m. for a 6 a.m. departure. No matter 
how automated we get the cockpit, until these scheduled duty 
times are addressed then there will always be stress and mistakes 
created by factors that only managers have control to change, we 
as pilots have tried for many years to correct this and still see 

it in the 90* s. 53 


Fatigue is a problem in flight crews of any aircraft type. The 
problem with automated cockpit crews is that they tend to rely 
too much on the aircraft when fatigued. For example, when I am 
tired I don't monitor altitudes in climbs and descents as closely 
as I should to ensure altitude captures. The point X am trying 
to make is that will all the technology available today, I f eel 
that more emphasis should be put on ways to provide safeguards in 
cockpit automation to assist fatigued crews. The MD-88 is 
definitely a step in that direction, but more should be done to 
increase its reliability. 56 


I retire in nine years. If the glass cockpits are the future, 

I'm glad to be retiring. Continuous automation does result m a 
deterioration of flying skills, and the big problem is what do 
you do when the electrons quit? Will a pilot still remember 
basic flying skills? I fly with copilots now who get upset and 
have difficulty handling the A/C if the flight director is off. 
They're hooked on the steering bars. Without them, they're lost. 
ATC is also a problem. A lot of new hire controllers know 
nothing about airplanes, and especially do not realize that 250 
kts. at 28,000 feet is not the same as 250 kts. at 10,000. I am 
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also receiving a lot of ambiguous clearances from controllers 
these days- I'm not sure what their standards are, but I know 
they need to be raised- The quality of new hire pilots is still 
qood, owing to my company's preference for ex-military types. 
Experience is still very important in this business, and it s the 
real reason the system isn't more dangerous than it is- 57 


The current training policy when switching aircraft is 
ridiculous. I was trained for the 767 from October 8 to November 
15, then I went back to the MD-88 from them till December 31. 

Then on January 7 I am on the 767 again. With the complex 
cockpits and systems of today's planes, do they really expect me 
to stay proficient in one plane while training m another. 58 

X feel increasing mental strain due to the legalistic climate 
today. Though I do my best personally and attempt to foster good 
communications and coordination with my fellow crew members and 
other company employees who are part of this team, it seems that 
this may not be enough to satisfy the FAA or the press should a 
decision of mine be questioned. For example, taxiing at BHM for 
flight to ATL, "autospoiler inop" messages is displayed on OAP. 

We stop, check the POM, reset the C/Bs as directed, but the 
message is still displayed. Next we check the MEL as directed 
for landing weight penalty and description of items to comply 
with in order to fly the aircraft with this message displayed. 
Requirements include: A) Do not arm autospoilers for T/O or 

landing? B) Do not arm autobrakes for T/0? C) Autobrakes must 
be placarded (MCO sticker applied) . Do we: 1) Return to the 

gate to have MTC apply placard? 2) Comply with A and B and 
write up malfunction, continue to ATL, and have MTC apply placard 
there? I feel that option 2 is no less safe than option 1, but 
in today's climate I chose to return to the gate. This all took 
place in a recent LOFT, so it didn't cost the company any money 
or inconvenience any passengers, but my concern is the effect on 
the system this sort of thing is going to have. 60 


The job today is not what it was 20 years ago when I was hired. 
Today there are many factors combining together to erode the 
satisfaction and esteem we once enjoyed in this career. 

Sometimes we pilots have been our own worst enemies. I've never 
heard a doctor tell people that open— heart surgery is a piece of 
cake . Yet pilots routinely play down the demanding tasks that we 
are called upon to perform. A Cat IX approach in a blizzard with 
cross winds, snow and ice covering the runway, and a generator 
inop - ''Aw shucks. Miss, there ain't nothing to it. Any monkey 
could learn this.” In my career I've been through a fuel crisis, 
controller strike, two recessions, and deregulation. That was 
nothing compared to the current frenzy over drug testing , 
security screening, leveraged buyouts, and checklist changes. We 
at this airline are fortunate that we have such a strong company. 
Yet we feel the trickle-down effects of these things none-the- 
less. The more automated the A/C becomes, the more my role 
chanaes from pilot to flight manager . The best system I've heard 
of is at Air Alaska. The bought head-up displays (HUD) for their 
727 's which allows pilots to hand fly a Cat III approach! In 
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summary, the airline pilot job is not what it used to be. But we 
all go through changes in life, so what's the bicf deal? Well, 
when frustration levels are increased at what poxnt is job 
performance adversely affected? 85 


DC-9 Crews 


You should have done this <experiment> prior to the manufacturers 
and airlines dumping millions into the development and purchase 
of all this fancy stuff. I prefer the close coordination of the 
2 -man crew. However, in emergency situations I would prefer 
three men (hijackings, emergency descent for structural problems, 
any type of situation that may require one person out of the 
cockpit) . Having one person left to work radios and possibly 
operate systems (especially abnormal systems) is too much for a 
safe operation. 4 

Look into the scheduling practices of various airlines. It seems 
that everyone wants to get his pound of flesh out of the crew; 
the company, the passengers, the news media, the FAA, the NTSB, 
the courts, the taxing authorities, airport security, and DOT 
(with the drug testing) . Pilots have to overlook all this 
pressure and still do a safe and professional job. All of this 
takes is toll mentally and physically. 6 

Scheduling trips with long days and short layovers and back side 
of the clock <night flights> that include landing in the early 
morning and flying another leg do not lead to a very safe 
operation. Too much time is being taken away from aircraft 
training and used for CRM and security trainxng. 24 

Pilots and flight attendants should be scheduled together. At 
least all day, but entire rotations or even an entire line for a 
month would help team building and allow the entire crew to work 
together much better. Sometimes we 1 11 see five crews a day. 
this does not make our job any easier. It would be nice to know 
the people we work with long enough to know their names. Another 
airline schedules crews together, so the M inefficiencies” 
argument doesn't hold a lot of weight. Without exception all of 
the F/As and pilots I have talked to agree with me. 26 

I feel that now the most irritating part of my job is the ATC 
environment. Specifically the lack of coordination that goes on 
between controllers. It seems that each controller sequences his 
traffic according to a letter of agreement rather than the next 
controller's ability to handle the aircraft. This is evident 
when you are switched to a new controller and he/ she asks why you 
are going so slow/ fast. 28 

F/Os need to realize that they have a ticket that can be taken 
away and a job that can be lost. If they are too meek to 
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criticize me when I am screwing up, they will have to pay the 
consequences. I try to point out that I'll be making mistakes 
and they must call them to my attention. No one pilot can do it 
all by himself. 38 

Why are we so tired after a three or four day trip? The day 
after a hard trip you feel almost poisoned. It's not a "good 
tired", such as results from hard manual work. Rather it is a 
harsh feeling that seems to go all the way to the 
I«ve never understood this. Our work is not hard physically and 
it's not all that stressful mentally. Find a cure for this and 
you can retire early. 39 


I think that more emphasis needs to be placed on studying 
physiological effects of poorly conceived flight rotations with 
continually changing sleep cycles, inadequate rest periods, and 
lack of opportunities for proper nourishment. Being tired, 
dehydrated and hungry probably plays a much larger role in crew 
effectiveness than the type of automation we employ. 41 


1) Lack of food on some flights with long duty days, or 
early/late flights when food service is closed. 2) Work days - 
early one day, late the next. More attention should be paid to 
diurnal cycle by scheduling. 3) Get the reporters (news) and 
lawyers out of the cockpit. Flight recorders (cockpit) are 
off limits until after all investigation is over and finalized. 
4) Spend FAA money to upgrade all runways used for commercial 
aviation, with a minimum of: VASI, R/W markers with 1000 foot 

length remaining, ILS, VOR/DME on all airports. The mone y held 
by the government in the Airport Fund <Aviation Trust Fund> to 
help balance the budget figures doesn't do much to help safety. 
49 


There has to be some way to improve communications between ATC 
and the airplane. A voiceless system using an up/ down link with 
visual screen and printout (if desired) would be great. Don t 
overload pilots with senseless, repetitious checklist items, 
briefings which are often not listened to. Callouts are for the 
benefit of the CVRs. All in all, we're a more standardized pilot 
group which is good. However, let's not cross over a common 
sense line and run a SAC strategic Air Command> type operation. 
It's still a fun job - let's keep it that way. 51 

In the Marine Corps in Viet Nam I sometimes flew over 90 hours 
per month. As a commuter airline pilot and flight instructor I 
flew nearly 100 hours some months. As a corporate pilot I flew 
about 35 hours a month. With this company I have flown all full 
contractual months with few exceptions for 14 years. My 
conclusion? 35 hours is about minimum for staying sharp, more 
than 80 is too much. Additionally, I believe that rotating 
shifts, disrupted sleep patterns, back side of the clock flying, 
etc. is highly deleterious and should be treated as such for 
scheduling purposes. 67 

In today's flying you almost need a lawyer in the jumpseat. Any 
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mistake (including the wrong foods for a positive drug result) 
can cause a loss of license. Flying is not so hard, but keeping 
up with all the rules and being afraid to make a "mistake can 
cause an upset stomach. I*ve never known of a pilot who t 
deliberately made an error. Yet there is a feeling of having the 
screws tightened more and more, so no possible human error can 
occur. That is a wrong assumption the on the part of the public, 
press, FAA, and the government. Working in a hostile environment 
is not all that much fun, and it*s very tiring, leading to 
possible "errors" on the pilot *s part. It is tough to keep 
awareness at 100% while always looking over your shoulder and 
being second guessed. 80 
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VI. 


DISCUSSION AND CONCLUSIONS 


A. INTRODUCTION 

This volume outlines the purpose, rationale , and methodology o £ 
this studv and reports some of the data collected in a simulat 

Is well as from a questionnaire. At this wrrtrng the 
analysis of most of the data generated by thji i simulator 
experiment has not been completed, and will be reported in ^ , _ _ 
subsequent volumes of this report. Accordingly, the experimental 
data reported here do not allow the authors to d ^aft conclusive 
results: this will have to await the analyses which will be 

reported in Volumes II and III, and summarized in Volume IV. 
Indeed the primary questions which prompted this study , a need 
to understand differences between traditional and advanced 
technology cockpits with respect to crew coordination and 
communication, will not be addressed m this volume. 

Nonetheless we shall summarize and comment on the information 
reported here, as we believe that even m its incomplete state, 
the data reflect on the basic questions and hypotheses that 
motivated this study . 


B. SIMULATOR (LOFT) RESULTS 

In this section we shall discuss the results of the LOFT 
experiment covered in detail in Chapter IV. The reader should 
keep in mind that the purpose of the LOFT experiment was not to 
pit one aircraft model against another and declare a winner. It 
would better be described as an opportunity to determine the 
relative strengths and weaknesses of a traditional a nda high 
technology version of the same basic aircraft. In short, it 
should be regarded as an exploration, not a contest. 

Had it been a contest, it would have been a somewhat unbalanced 
one, for as we have pointed out in Chapter II, the DC-9 cr ®?[s 
were considerably more experienced m their aircraft than those 
in the MD-88. Although the DC-9 pilots had more flying time in 
their aircraft, both with respect to hours in type and months in 
type (see Chapter II), their total flying time was actually 
slightly less than the MD-88 crews (the difference being 
statistically non— significant) • Also, statistical analyses 
within fleets found no correlation between performance measures 
and time- in- type. 

The LOFT scenario described in Chapter III was designed to b© 
both realistic and fair to all crews. That is, we attempted to 
construct a LOFT that would not, in any way that we could 
predict, favor either the traditional or the high— technology 
aircraft. The electrical systems that were failed and the 
abnormal procedures for combating the failures were virtually 
identical in the two aircraft; the other problems dealt with 
weather and decision-making, which was essentially aircraft type 
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independent . 

If there was one factor we attempted to build in, it was that the 
LOFT be communication-* intensive . We believe that we accomplished 
this. The LOFT was highly demanding of the crews, but within the 
bounds of realism, and appeared to meet the goal of requiring a 
highly coordinated effort between the two pilots. The 
combination of the electrical problems, deteriorating weather 
both at the destination and potential alternates, and clearance 
into a somewhat irregular unpublished holding pattern created a 
high workload atmosphere throughout the flight, and the need for 
a well coordinated cockpit. A "one-man show" probably could not 
have completed the LOFT successfully. 

The fact that the LOFT scenario was communication- intensive 
provided t h ose crew members who had been ^ through the company s 
new CRM program the opportunity to practice what they had 
learned. Even those who had not yet taken the course were 
certainly aware of it, and mindful of the lessons that they would 
soon learn. We will have to await the publication of Volume III 
to gain insight into how the crews of the two very different 
aircraft handled their cockpit communications demands. 


Workload 

One of the significant differences produced by the LOFT data was 
that MD-88 pilots, using the NASA Task Load Index (TLX) rated 
their own workload as higher than did the DC-9 pilots. This 
finding is somewhat consistent with the information obtained on 
the questionnaire (see Chapter V, Section D, particularly open- 
ended question No. 5 on workload. See also open-ended question 
No. 7, for pilots who have flown both a traditional and a glass 
cockpit aircraft.) Although there was great diversity of 
opinion, it was generally felt that in spite of some of the 
obvious advantages of the glass cockpit (e.g. the HSI map mode 
display, the superior autothrottle, etc.), the workload m the 
MD-88 could become excessive. 

This was also consistent with the findings of previous field 
studies in automation (Curry, 1985? Wiener, 1985, 1989), where 
pilots of the advanced technology aircraft expressed the feeling 
that during periods where the workload was high, the automation 
increased the workload, and where it was low, it tended to reduce 
workload. 

The concern over workload in the modern cockpit was also 
consistent with opinions of crew members in the host airline's 
757/767 and MD-88 fleets whom the first author spoke with during 
jumpseat observations. The typical comment, heard repeatedly 
during interviews, almost to the word, was "I love this airplane, 
love the power and the wing, and I love this stuff (pointing 
toward the mode control panel and CDU) , but I've never been so 
busy in my life." At this point they often add, "But some day 
it (automation) is going to bite me." The word "busy" and the 
sentiments just quoted appear over and over in the lexicon of the 
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glass cockpit pilots, even though as seen above, their overall 
view of the aircraft is highly favorable. 

Just as the word '•busy" appeared so often in the conversations 
and open-ended questionnaires of MD— 88 pilots, the word heard and 
seen over and over in the opinions of DC-9 crews was "simple 
(and "simplicity") . The DC-9 pilots repeatedly described their 
aircraft, its systems and operations, as "simple." (Note: many 

of the attitude items reported in Chapter IV address the question 
of workload and automation.) 

The reasons for the subjective feeling of excessive workload in 
the modem technology aircraft has been explored m the previous 
field studies (see references above) , and need not be repeated 
here. We would mention that the LOFT portion of this study 
essentially validated both the problems and strengths of cockpit 
automation reported in previous NASA research, and gave emphasis 
to the words of previous authors (Wiener and Curry, 1980) 
regarding the necessity to allow pilots to work out their own 
solutions to problems, within reason, employing automation, or 
turning it off, as they see fit. 

An interesting example was the entry into the holding pattern at 
Columbia. This was a demanding procedure, which was required by 
ATC due to the presence of a no-radio (NORDO) aircraft m the 
vicinity of Columbia VOR (CAE) . The crews were just coming off 
of a missed Category II approach, and had to copy their clearance 
and plan their entry and holding pattern. For the DC-9 crews, 
there seem to be few alternatives and little trouble. They 
simply dialed the radial on their VOR course selector and with 
this and DME flew by basic airmanship into the holding pattern. 

For th e MD— 88 crews, there were alternatives. They could fly the 
entry using basic airmanship and VOR navigation, much as the DC-9 
pilots had done, or they could turn to the CDU holding page, set 
up a waypoint to be used as the holding fix, and fly via direct 
intercept to the holding fix, then allowing the FMC to establish 
their holding pattern, and hold automatically. Many of the MD-88 
crews that attempted this did so correctly, and exploited the 
full benefits of automatic holding. However, several crews had 
difficulty entering the unpublished holding pattern into the CDU, 
or got "behind the airplane" and could not catch up, and got lost 
on the entry. 

Perhaps the smoothest and safest performance of this task came 
from those MD-88 crews who combined basic airmanship with 
automation. These crews elected to ignore the hold page and FMC- 
based navigation in general , and manually selected the VOR and 
radial much the way the DC-9 pilots had done it. They then flew 
manually or at a lower level of automation (e.g. autopilot, 
heading select, and altitude hold). Upon crossing the holding 
fix and establishing their inbound leg, they then turned to the 
hold page and assigned the FMC the task of establishing and 
maintaining the holding pattern. 
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This type of performance, with a crew (presu m a b ly the pilot 
flying) deciding when to use automation and when not to, has been 
an issue in management and training in the last decade. One 
extreme policy has been to require that the crews utilize the 
automation. Pilots in previous field studies have referred to 
this management policy as "We bought it, you use it." Many 
pilots in this and previous automation studies have been critical 
of their transition training for an alleged over-emphasis on 
automation, on teaching only how to use, rather than whether to 
use, various features of the available automation (see open-ended 
question No. 3) . 

Curry (1985) urged training departments to offer what he called 
"turn- it -off training." It would appear to the authors that this 
particular portion of the LOFT experiment, and the various 
solutions attempted by the crews, brings this problem into focus, 
and validates the recommendations of Wiener and Curry. About a 
year after the LOFT study, two airlines. Delta and Federal 
Express, promulgated their own "Philosophy of Automation" (see 
Appendix 4) . Delta's philosophy is clearly consistent with the 
view that flight crews should have discretion, within the bounds 
of reason, to use or not use automation, consistent with their 
views of the best and safest way to perform a maneuver. 

Perhaps out of these studies and the examination of the question 
by Delta, Federal Express, and other airlines will emerge a new 
doctrine: "We bought it, you decide if and when you wish to use 

it." We would endorse such a management policy. 

Crew Performance Ratings 

The three measures of crew performance as judged by the expert 
raters, the LOFT instructor and the NASA obseiprer, produced few 
statistically significant contrasts. Where significant results 
were found, they again favored the DC-9 crews over the MD-88 
crews. The crew composite ratings from the Overall Rating Form 
filled out by the NASA observer revealed significantly higher (18 
per cent) ratings for the DC-9 crews. On the Detailed Rating 
Form the DC-9 crews again were seen as superior, though the 
difference (about 8 per cent) fell just short of statistical 
significance. On the CRM Evaluation form (sometimes called 
"Line/LOFT" form) no differences between aircraft models were 
detected. The interpretation of these between— model differences 
will be discussed later in this chapter. 

It is interesting that so few contrasts produced significant 
differences on the crew rating scales. It is not easy to 
interpret the failure of the captain versus first officer as 
pilot flying to produce differences. There was in our LOFT plan 
a defect that may have diminished somewhat the potential of this 
variable. At this airline the first officers are not permitted 
to perform the duties of PF on Category II approaches. As a 
result, at the point in the flight to Columbia where it was 
ascertained that the weather required a Category II approach, m 
those crews in which the first officer had been the PF, the 
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captain had to take back the PF role for the approach. Following 
the missed approach, some captains gave it back to the first 
officer, but most elected to remain as PF for the holding pattern 
and the diversion to Charlotte. 

As a result of this transfer of duties, our ability to assess 
differences in crews with the duties were assigned to captain or 
first officer suffered during one of the critical points of the 
flight. Differences between these conditions may emerge from our 
analyses of crew coordination and co mmun ication ( CRM) , which will 
be covered in Volume III. 

In an analysis of reports to NASA's Aviation Safety Reporting 
System (ASRS) , Orlady (1982) found not a superiority in one or 
the other conditions, but the fact that certain types of errors 
prevailed under each condition. For example, when the captain 
was the PF, there was a greater number of reports of near mid-air 
collisions (NMACs) , takeoff anomalies, and crossing altitude 
deviations. When the first officer served as PF, there were more 
altitude deviations, NMACs during approaches, and landing 
incidents. Orlady' s study was completed before the appearance of 
the current generation of high-technology aircraft, and we do not 
know of any study of the combined effects of PF/PNF duty 
assignment and cockpit automation. 

Crew Errors 

In spite of the higher performance ratings of the DC-9 crews, no 
such differences could be found in our analyses of error data. 

Our experimental hypothesis, based on the field studies of Wiener 
and Curry, including interviews and questionnaires by pilots of 
advanced technology aircraft, was that cockpit automation 
generates more serious errors than traditional technology. Thus 
we expected to see, among crews flying the same LOFT scenario, 
not necessarily a difference in the number of errors, but a 
different error distribution, namely for the MD-88 crews a higher 
frequency of the more serious errors in our three-way 
categorization of error severity. The reasons for this 
hypothesis are discussed in Wiener and Curry (1980) , and Wiener 
(1988, 1989) . To put it very briefly, it is in the nature of 
digital systems to invite more serious errors, due primarily to 
the opportunity to make blunders in digital input, but due also, 
to the relative lack of "coupling" of the pilot to the machine in 
the highly automated models, and the lack of feedback inherent m 
many automated systems in aviation and elsewhere. 

While Wiener and Curry felt that they had some basis for such a 
prediction, still there had not been an experimental evaluation 
of the types and severity of errors that would be produced by the 
two levels of technology. We believe that this is the first 
attempt to verify experimentally the automation-severity of error 
hypothesis. As the data reported in Chapter IV . indicates , we 
failed to find support for the hypothesis in this experiment. 
Table IV-4 and the analysis of variance of these data reveal no 
differences in mean number of errors between DC-9 and MD— 88 
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flights, nor is there a significant difference in the 
distribution across the three severity classifications. Likewise 
the error analysis performed by the NASA observer, who rank- 
ordered the severity of the errors committed by DC-9 and MD— 88 
crews, showed that the mean error severity of the two aircraft to 
be essentially equal. 

It is true that we found no support for our hypothesis that the 
more automated aircraft would generate more severe errors. 
However, it is equally important to take note of the fact that 
the opposite hypothesis, that automation eliminates human errors 
by eliminating their source, oft stated by the producers and 
supporters of automation, found no support from this experiment. 
It is long been the dream of traditional engineers to "automate 
human error out of the system. " To the degree that this 
experiment represents a realistic simulation of a highly 
demanding airline flight segment, we find little to comfort those 
who see automation as a cure for human error. 

C. CONCLUSIONS 

What does this say for automation? Granted that the MD-88 crews 
were less experienced in their aircraft, it appears that where 
differences exist, they favor the performance of the 
traditional technology air crews. As to the difference in 
experience levels, our sample of pilots actually represent the 
real world of line flying, in that for at least the next few 
years, the pilots of advanced technology aircraft will have less 
time in type than those flying traditional aircraft. The MD-88 
crews did, however, have equal (actually slightly higher) total 
flying time than the DC-9 pilots. In the years to come, these 
differences will be less apparent: the advanced technology 

aircraft will no longer be the "odd balls" of the fleet, but will 
be the mainstay. And accordingly, their crews will have time m 
type approaching that of the pilots in the older . aircraft. We 
say "approaching" because for several years, until the 
traditional planes are retired, the phenomenon of DC-9, B-737-100 
and -200, and particularly B-727 pilots, who spend many years in 
the same cockpit and accrue a vast amount of time in type, will 
remain. The 20-year B-727 veteran can be found at almost any 
airline. 

Setting aside the differences in experience in type, we ask why 
advanced technology has not served its pilots better? 

The pattern of results provide only weak evidence of either 
higher workload or poorer performance among the MD— 88 crews. The 
statistical contrasts critical to our experimental hypothesis 
failed to provide statistically significant differences. Analysis 
of the observer's detailed ratings only approach traditional 
statistical significance criteria, while instructor ratings and 
errors are in the same general direction but are nonsignificant. 
We have identified either an overall effect of substantially 
lower magnitude than those seen in previous investigations or a 
problem that occurs only in narrowly defined situations. 
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We would have to conclude that, with the except °£ 
ratines of crew proficiency, we have failed to find report aoie 
difference between the performance of DC-9 and MD-88 crews, and 
thus at this point cannot draw any inferences about the effect of 
cockpit automation on the ability of qualified crews to fly a 
difficult mission. We have failed to produce evidence to support 
our hypothesis that automation would generate more serious 
errors. Clearly one may also conclude that we have not produced 
a case in favor of high technology cockpits --- that the crews of 
the DC— 9, a product of mid— 1960 decade technology, performed just 
as well as those flying a very advanced, very expensive, modern 
technology aircraft. 


D. EPILOGUE 

Aviation safety is a living, growing, constantly changing 
enterprise. Times change, new problems emerge, new equipment 
appears, and the industry enjoys a steady improvement in 
machines, materials, training methods and devices, maintenance, 
information, procedures, supervision, and management. Many or 
the problems pointed out in this report and in subsequent volumes 
in this series have already been remedied. 

Certain portions of this report, mainly the discussion of errors 
made both on line flights and in our simulation experiment, 
involve self-criticism. It is a testimonial to the dedication of 
the management of the cooperating airline and the professionalism 
of the volunteer pilot group that they would share their 
experiences and opinions with the authors, and hence with the 
aviation community at large. 

The errors and weaknesses reported here should not be viewed as 
criticism of the host airline and its pilots , but as an example 
to others of the unselfish efforts on the part of a company and 
its employees to understand and improve the human factors of 
airline flight. The willingness to examine, recognize, and 
report conditions that require remedy is the foundation of flight 

safety . 
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APPENDIX 1 - RATING FORMS 


Form 


Filled 
out by 


Participant Survey 
Overall Rating Form 
Detailed Rating Form 
CRM Evaluation Sheet 


Each pilot 
NASA observer 
NASA observer 
LOFT instructor 
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ID Numbe r 

Participant Survey 

Please fill out this short survey to give us your opinion about the LOFT scenario you have just 
completed. 

Crew Position (circle one) CAPT FO 


1. Were you aware of the events and problems presented in this LOFT prior to participating today? 
No information about any aspect of it. 

Slight familiarity with problems and events. 

Considerable familiarity with problems and events. 

Detailed information on problems and events. 

Please evaluate the work or effort required by the scenario using the scales below. A general 
definition is provided for each scale. 

2. Mental Demand: How much mental and perceptual activity was required (e.g., thinking, 

deciding, calculating, remembering, looking, searching, etc.)? Was the task easy or 
demanding, simple or complex, exacting or forgiving? 

Low 1 2 3 4 5 6 7 High 

3. Physical Demand: How much physical activity was required (e.g., pushing, pulling, 

turning, controlling, activating, etc.)? Was the task easy or demanding, slow or brisk, 
slack or strenuous, restful or laborious? 

Low 1 2 3 4 5 6 7 High 

4. Temporal Demand: How much time pressure did you feel due to the rate or pace at which the 

tasks or task elements occurred? Was the pace slow and leisurely or rapid and frantic? 

Low 1 2 3 4 5 6 7 High 

5. Performance: How succe ssful do you think you were in accomplishing the goals of the task 

provided by the LOFT? How satisfied were you with your performance in 
accomplishing these goals? 

Perfect 1 2 3 4 5 6 7 Failure 

6. Effort: How hard did you have to work to accomplish your level of performance? 

Low 1 2 3 4 5 6 7 High 

7. Frustration Level: How insecure, discouraged, irritated, stressed, and annoyed versus 

secure, gratified, content, relaxed, and complacent did you feel during the task? 

Low 1 2 3 4 5 6 7 High 
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OVERALL RATING FORM 


Crew Number/Date / 

Aircraft DC-9 MD-88 (circle one) 


Evaluate the crew as a team on each of the following items: 
CREW COMMUNICATIONS AND DECISION MAKING 

1. Communications were thorough, addressing 

coordination, planning, and problems anticipated Not at all 1 2 3 

2. Open communications were established among 

crewmembers. Not at all 1 

3. Timing of communications was proper. Not at all 1 

(^Communications were relevant, complete, and 
^verified. Not at all 1 

5. Active participation in decision making process was 

encouraged and practiced. Not at all 1 

6. Alternatives were weighed before decisions were 

rnqrjp. final. Not at all 1 

(^Appropriate immediate actions were taken when time 
v "*was not available for crew decision making. Not at all 1 


2 

2 

2 

2 

2 

2 


4 5 

4 5 
4 5 

4 5 

4 5 

4 5 

4 5 


Very much 

Very much 
Very much 

Very much 

Very much 

Very much 

Very much 


INTERPERSONAL (MANAGEMENT) STYLES AND ACTIONS 

8. Crewmembers showed concern with accomplishment 

of tasks at hand. Notatall 1 2 3 4 5 Very much 

9. Crewmembers showed concern for the quality of 

interpersonal relationships in the cockpit Notatall 1 2 3 4 5 Very much 

WORKLOAD AND PLANNING 
0) Overall workload 

. Work overloads were reported and work prioritized 
or redistributed. 

12. Crewmembers planned ahead for high workload 
situations. 

13. Appropriate resources were used in planning. 


Low 

1 

2 

3 

4 

5 

High 

Not at all 

1 

2 

3 

4 

5 

Very much 

Not at all 

1 

2 

3 

4 

5 

Very much 

Notatall 

1 

2 

3 

4 

5 

Very much 


CREW ATMOSPHERE AND COORDINATION 


14. Overall vigilance 

15. Interpersonal climate 

16. Preparation and planning. 

17. Distractions avoided or prioritized 

lg. Workload Distributed and communicated 
Conflict resolution 
® Overall technical proficiency 
21. Overall crew effectiveness 


Inattentive 

1 

2 

3 

4 

5 

Alert 

Hostile 

1 

2 

3 

4 

5 

Friendly 

T fit? 

1 

2 

3 

4 

5 

Well in Advance 

Poor 

1 

2 

3 

4 

5 

Excellent 

Poor 

1 

2 

3 

4 

5 

Excellent 

Poor 

1 

2 

3 

4 

5 

Excellent 

Poor 

1 

2 

3 

4 

5 

Excellent 

Poor 

1 

2 

3 

4 

5 

Excellent 


Items 4, 7, 10, 19 and 20 excluded from analyses 
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DETAILED RATING FORM 

Crew Number/Date / 

Aircraft DC-9 MD-88 (circle one) 


Evaluate each crewmember. Use the following ratings for all categories: 



1 - below average 

2 - slightly below average 

3 - average 

4 - slightly above average 

5 - above average 




n/a - not observed or not applicable 


Items specific to a phase are indicated in bold. 



PRESTART 

Captain 

First Officer 

Notes 

APU Problem Resolution 

1 2 3 4 5 n/a 

12 3 45 n/a 


Crew Coordination/Communications 12 3 4 5 n/a 

12 3 4 5 n/a 


ATC Sc Ground Communications 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Plan. Sc Sit Awareness 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Procedures, Checklists, Callouts 

1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


Overall Performance Sc Execution 

1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


TAXI/TAKEOFF 

Captain 

First Officer 

Notes 

Crew Coordination/Communications 1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


ATC Communications 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Plan. Sc Sit Awareness 

1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


Procedures, Checklists, Callouts 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Aircraft Handling 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Overall Performance & Execution 

1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


CLIMB 

Captain 

First Officer 

Notes 

Crew Coordination/Communications 1 2 3 4 5 n/a 

12 3 4 5 n/a 


ATC Communications 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Plan. & Sit Awareness 

1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


Procedures, Checklists, Callouts 

1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


PA & PAX Handling 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Aircraft Handling 

1 2 3 4 5 n/a 

12 3 4 5 n/a 

- 

Overall Performance Sc Execution 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


CRUISE 

Captain 

First Officer 

Notes 

Crew Coordination/Communications 1 2 3 4 5 n/a 

12 3 4 5 n/a 


ATC Communications 

1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


Plan. & Sit Awareness 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Descent Planning 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Procedures, Checklists, Callouts 

1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


PA & PAX Handling 

1 2 3 4 5 n/a 

12 3 4 5 n/a 


Aircraft Handling 

1 2 3 4 5 n/a 

1 2 3 4 5 n/a 


Overall Performance Sc Execution 

1 2 3 4 5 n/a 

12 3 4 5 n/a 
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A 


DELTA AIR LINES CRM EVALUATION SHEET >SikS 

DELTA 

AIH LINKS 

SECTION I: PERSONNEL DATA: 

Rater ID # ' Rater yr. of birth _________ 

Date: Month Year (Do NOT enter the day) 

Number of previous CRM seminars? Capt. _______ F/O _____ 5/0 

Number of previous LOFTs? Capt. F/O ______ 5/0 ________ 

Base: Capt. F/0 S/O Equipment ___ 

Please place a checkmark to identify the crew member(s) and type(s) of training or checking being evaluated. 
Line Check for Capt. F/O S/0 or Crew Audit ______ 


Number of Takeoffs and Landings observed _ 
Total Amount of time observed 

r 30 min) followed bv training. LOFT # 


(Line Check Only) 

(Line Check Only) 


LOFT (2 hr 30 min) followed by training. LOFT # 

Captain PC + 45 min. CRM LOFT with special crew training. LOFT # 
Other training (specify type and position) 


SECTION II: 


CREW RESOURCE MANAGEMENT EVALUATION 


Briefing: Thorough, establishes open communications, addresses 
coordination, planning, team creation, and anticipates problems 

Poor 

1 

(CIRCLE) 
2 3 4 

5 

Excellent 

Communications: timely, relevant, complete, and verified 

Poor 

1 

2 

3 

4 

5 

Excellent 

Inquiry/questioning practiced 

Poor 

1 

2 

3 

4 

5 

Excellent 

Assertion/advocacy practiced 

Poor 

1 

2 

3 

4 

5 

Excellent 

Decisions communicated and acknowledged 

Poor 

1 

2 

3 

4 

5 

Excellent 

Crew seif-critique of decisions and actions 

Poor 

1 

2 

3 

4 

5 

Excellent 

Concern for accomplishment of tasks at hand 

Poor 

1 

2 

3 

4 

5 

Excellent 

Interpersonal relationships/group climate 

Poor 

1 

2 

3 

4 

5 

Excellent 

Overall vigilance 

Poor 

1 

2 

3 

4 

5 

Excellent 

Preparation and planning for inflight activities 

Poor 

1 

2 

3 

4 

5 

Excellent 

Distractions avoided or prioritized 

Poor 

1 

2 

3 

4 

5 

Excellent 

Workload distributed and communicated 

Poor 

1 

2 

3 

4 

5 

Excellent 

Overall workload 

Low 

1 

2 

3 

4 

5 

Hiflh 

Overall TECHNICAL proficiency 

Poor 

1 

2 

3 

4 

5 

Excellent 

Overall CREW effectiveness 

Poor 

1 

2 

3 

4 

5 

Excellent 

If Observed: 

Management of abnormal or emergency situation 

Poor 

1 

2 

3 

4 

5 

Excellent 

Conflict resolution 

Poor 

1 

2 

3 

4 

5 

Excellent 


APPENDIX 2 - ATTITUDE QUESTIONNAIRE 


ID Code: 

Please place your self -assigned NASA ID code above. If you do 
not know your code, just attach a piece of paper with your name. 
We will replace your name with the code and destroy the paper. 
Once all the questionnaires are in, we will destroy the only ID- 
to-name list once and for all. Where we use the shorthand 
DC-9/MD-88, take that to mean whichever model you currently fly. 

If your aircraft or seat assignment changed since you joined the 
study last summer, what is your current aircraft and seat. Reply 
only if there has been a change. 

aircraft seat — 


I . AIRCRAFT EXPERIENCE 

We would like to know your past experience in your company's 
turbojet aircraft. Please consider your experience only at this 
company. Place an "X” in the box for each seat on each aircraft 
that you have ever occupied. Do not put flying time. 


DC-9 
MD-88 
B— 737 
B— 757/756 
B— 727 
DC- 8 
L— 1011 
DC-10 
B— 747 


SEAT 


Captain 


* 

* 

* 

.* — - — - — 
* 

* 

* 

.* — — 

* 

.* — — — — — 
* 

* 


*- 

* 

*- 

* 

*- 

* 

*■ 

* 

* 

* 

.* 

* 

.* 

* 

.* 

* 

.* 


F/O 


*. 

* 

*- 

* 

* 

* 

* 

* 

* 

.* 

* 

.* 

■k 

.* 


S/0 

xxxxxxxx 


xxxxxxxx 


xxxxxxxx 


xxxxxxxx 


•k 
k 
k 
k 
k 
k 
* 
* 
.* 
* 
■ k 
k 
.* 
k 
.* 
k 
.* 
* 
.* 


1. Which seat in which aircraft did you occupy immediately 
before going to DC-9/MD-88 school? 

Aircraft Seat 
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2. Approximate total flying hours at this company (include S/0) 

hours 


3. Approximate hours DC-9 


MD-88 


3a. How many months has it been since your DC-9/MD-88 
transition to your present seat? 


months 


4. What do you consider the most advanced aircraft (with respect 
to instrumentation, avionics, automation etc. that you have 
flown? Include military or other employers: 

ans: 


5. Do you own a home computer? (Y/N) ans: _ 

6 If yes, what type? 

7. MD-88 ONLY: Please indicate the approximate number of 

autolands you have made (either as PF or PNF) in the last 
year (1989) . 

Number 

8 . Approximately how many actual Cat II or III approaches have 
you made (as PF or PNF) in the last year (1989)? 

Cat II ? Cat III (MD-88 only) 

9. Approximately how many non-precision approaches (as PF or 

PNF) have you made in the last year (1989)? 

VOR LOC ADF 


10. If the money and quality of trips were all the same, what 
would be your first choice of plane to fly in company's 
present fleet? 


Aircraft: 
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II . attitude-toward-automation scale 

This is a 28-item attitude scale. It is called an "intensity 
scale" because you can indicate not only your agreement or 
disagreement with the statements, but the extent to which you 
agree/ disagree. Note that the statements can be positively . or 
negatively stated. The scale is straight-forward — there is no 
attempt to be "tricky." 

Answer all guestions based on your present aircraft. That is, if 
you now fly the MD-88, but once flew the DC-9, answer as an MD-88 
pilot. Where we use the shorthand "DC-9/MD-88" , we mean 
whichever one you are currently flying. 

For the purpose of these questions, consider the word 
"automation" to mean autopilots, autothrottles , flight directors 
etc. in the DC-9 7 and in the HD-88, the more advanced flight 
guidance and controls as well. 


1. Flying today is more challenging than ever. 

2. I take active measures to prevent a loss of my flying skills 
due to too much automation. 

3. The DC— 9/MD— 88 automation works great in today* s ATC 
environment . 

4. It is important to me to fly the most modern plane in my 
company's fleet. 

5. As I look at aircraft today, I think they've gone too far 
with automation. 

6. I always know what mode the automation is in. 

7 . I use the automation mainly because my company wants me to . 

8. In a highly automated plane, you run the risk of loss 
of basic flying skills. 

9 . Automation frees me of much of the routine , mechanical parts 
of flying so I can concentrate on "managing" the flight. 

10. I am not concerned about making errors, as long as we follow 
procedures and checklists. 

11. I look forward to more automation - the more the better. 

12. I have no trouble staying "ahead of the plane". 

13. CRM training is more important for two-pilot than three- 
pilot crews. 

14. Automation does not reduce total workload. 
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15. It is easy to bust an altitude in today’s environment. 

16. Flying the DC-9/MD-88 in congested terminal areas such as 
Washington and New York is not particularly difficult. 

17. Training for the DC-9/MD-88 was as adequate as any training 
that I have had. 

18 . I am concerned about the reliability of some of the 
automation equipment . 

19. I prefer the two-pilot cockpit to the three— pilot operation. 

20. I am concerned about the lack of time to look outside the 
cockpit for other aircraft. 

21. I use automation mainly because it helps me get the job 
done. 

22. Our CRM training has been helpful to me. 

23. Some times I feel more like a ’’button pusher" than a pilot. 

24. There are still modes and features of the DC-9/MD-88 
automation that I don't understand. 

25. There is too much workload below 10,000 feet and in the 
terminal areas. 

26. In the DC-9/MD-88, it is easy for the captain to supervise 
the first officer. 

27. Autoland capability enhances safety. 

28. Electronic flight instruments ("glass cockpits”) are a big 
advance in safety. 
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Ill . OPEN-ENDED QUESTIONS 


Note: In the original forms sent to the volunteers, 

allowed between the questions. For brevity this page 
has been condensed in this report. 


space was 
of the form 


Please answer the following questions in your own words. 


1. Describe in detail a error which you made, or observed, in 
the DC— 9 /HD— 88 that could have led to an incident or 
violation. How could it have been avoided? (equipment 
design? training? crew coordination?) . Please describe 
specifically what occurred. 

2. What can you say about crew coordination on the DC— 9/MD— 88? 
What areas of crew coordination/communication need 
improvement? 

3. What did you think of your training for the DC-9/MD-88? What 
topics should receive more/ less emphasis? Any comments on 
training aids and devices that were used, or needed? 

4. Do you like the way the DC-9/MD-88 operates in the ATC 
environment? Please mention things you have trouble with, 
and things that work well, in dealing with ATC. 

5. What can you say about the overall workload of the DC— 9/MD— 88 
compared to other aircraft you have flown? Include mental 
workload, monitoring etc. 

6. Please list the features of modes of your aircraft flight 
guidance^ instrumentation, and avionics that you like and 
dislike. 

Like Dislike 


7. If vou have flown both a trad itional and a glass cockpit 

aircraft (MD-88 or 757/767) , please give us your insights^ 
into the differences between operations in the traditional 
and the glass cockpit. (If not, please leave blank) . 

8 Please feel free to add anything else that you think we 
should know about the human factors of your flying job. 
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APPENDIX 3 “ SIGN-UP FORM FOR VOLUNTEERS 


Name 


Capt. or F/O? 


Address 


City 

State ZIP 

Home Phone: Area Code ( ) Number 

Are you presently an instructor or check airman? 

How long have you been flying the MD-88 ? month! 

Total MD-88 flying time hours 

Total flying time (including flight engineer) , all company 
aircraft: hours 


Make up an ID code for yourself and enter it below. Use any 
combination of letters and numbers (up to a max of 6) . Do not 
use your Social Security or company pay number, birth date, etc. 
Insert it in the blank below. The last two positions are for our 
use. The last indicates aircraft type (DC-9 vs. MD-88) . In the 
next-to-last , insert "l" if you are a captain, " 2 " if you are a 
first officer. 

The red sticker is for you. Please enter Y our f u il eight- 
character ID and keep it some place convenient. We suggest a log 
book or Jep manual. If you have questions, please call the 
Project Director or your Safety Committee. 

Your ID code: 
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APPENDIX 4 - DELTA AIR LINES AUTOMATION PHILOSOPHY STATEMENT 


The word "Automation", where it appears in this statement, shall 
mean the replacement of human function, either manual or 
cognitive, with a machine function. This definition applies to 
all levels of automation in all airplanes flown by this airline. 
The purpose of automation is to aid the pilot is doing his or her 
job. 

The pilot is the most complex, capable and flexible component of 
the air transport system, and as such is best suited to determine 
the optimal use of resources in any given situation. 

Pilots must be proficient in operating their airplanes in all 
levels of automation. They must be knowledgeable in the 
selection of the appropriate degree of automation, and must have 
the skills needed to move from one level of automation to 
another. 

Automation should be used at the level most appropriate to 
enhance the priorities of Safety, Passenger Comfort, Public 
Relations, Schedule, and Economy, as stated in the Flight 
Operations Policy Manual. 

In order to achieve the above priorities , all Delta Air Lines 
training programs, training devices, procedures, checklists, 
aircraft and equipment acquisitions, manuals, quality control 
programs, standardization, supporting documents, and the day-to- 
day operations of Delta aircraft shall be in accordance with this 
statement of philosophy. 
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APPENDIX 5 - PREVIEW OF VOLUMES II, III AND IV 


Listed below is a brief descriptions of forthcoming volumes of 
this project# Expected publication date is mid to late 1992. 

VOLUME II (Everett Palmer , Editor) 

This volume will consist of two parts: 

1. An extension of the analyses of crew error made during the 
LOFT flights. 

2 . An examination of the procedures as carried out by the LOFT 
crews, and an analysis of workload peaks. 


VOLUME III (Barbara G. Kanki , Editor) 

Volumes I and II are concerned with flight crew performance, and 
the relationship between performance and level of automation. In 
contrast. Volume III deals with group processes, or the means by 
which flight crew performance is achieved. We may think of level 
of automation as an input variable, and crew performance as an 
output variable? Volumes I and II attempt to express output as a 
function of input. Group process variables may be thought of as 
mediators between input and output. Mediating processes describe 
the strategies or behavior styles which differentiate among 
performances and typically involve communication patterns which 
are analyzed as dynamic, interacting sequences of behavior of 
flight crew member. Volume III will chart this mediation 
activity by a detailed analysis of crew communications from the 
LOFT exercises. It will explore other group processes related to 
communication skills, decision making, problem solving, and 
training countermeasures associated with effective crew 
performance . 


VOLUME IV (Earl L. Wiener, Editor! 

Volume IV will be an executive summary of the three previous 
volumes. It will highlight the findings, implications, and 
applications of the study. It will be relatively brief, and will 
be able to serve as a stand-alone summary of the study . 
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